Cracking the Evidence Conundrum: Four Ideas to Get People to Use Evidence

This visual summarizes findings from a recent literature review on evidence-based decision-making. This work is supported by the Global Development Lab (the Lab). We will be sharing the full report here once it is available. If you are interested to learn more, please contact the authors or Jessica Lucas at the Lab.

It is fairly obvious that using evidence to inform our decisions makes sense. USAID’s update to its operational guidance includes evidence-based decision-making as one of its key principles and collaborating, learning and adapting (CLA) as an approach to operationalize this. But using evidence in our daily lives is easier said than done, so what is stopping us from using evidence?

“USAID’s decisions about where and how to invest foreign assistance resources should be based on analysis and conclusions supported by evidence.” (ADS 201 series, pg. 11)

• Do we even have the "right" evidence to inform our decisions? A 2017 GAO study found that only around 15 percent of USAID managers said they agreed to a very great extent that they had access to the performance information they need to manage their operations or technical work. This is actually in line with the rest of the federal government agencies surveyed, but arguably it could be a lot better.
• Does the evidence conform to our pre-existing beliefs? Even when the evidence is available, if it doesn’t conform to what we already believe to be true, we may not find it credible and might be less likely to use it.
• Do political or organizational considerations outweigh the evidence? By and large, evidence isn’t the only input into a decision. What political or organizational imperatives take priority over the evidence? Sometimes those imperatives are in conflict with the evidence (example: working in a certain geographic area because of a local government request even though that area has less need from a development perspective).

Despite these barriers, there are some insights from research on evidence-based decision-making that can help us be more intentional about applying evidence to our decisions: To read the full literature review this is drawn from, click here. 

Insight 1: Engage and collaborate with decision makers in the process of generating and analyzing evidence. The literature suggests that while including decision-makers early and often in the process can be useful, outputs should be tailored to the decision-maker in mind. It suggests understanding the motivations of decision-makers, as well as the costs and benefits of interactions between evidence generators and users, to ensure engagement is feasible and effective. Moreover, there is evidence that techniques such as social influence, mentoring, and online engagement can be effective engagement tools.

Insight 2: Embed the use of evidence into existing organizational structures and processes. One way is to tie or integrate interventions together, such as integrating decision-makers in the evidence gathering process as well as providing communications about and access to evidence. Also, the literature suggests linking evidence to policy priorities and goals, and linking the use of evidence to planning, budgeting, and reporting processes, in order to integrate the use of evidence into how organizations operate. For inspiration, the OMB’s Analytic Perspectives, Budget of the U.S. Government, Fiscal Year 2018 (Washington, D.C.: 2017), as an example, has recommended that federal agencies “adopt a “learning agenda” in which they collaboratively identify the critical questions that, when answered, will help their programs to be more effective, and to plan to answer those questions using the most appropriate tools.”

Insight 3: Share the evidence: proactive in transmitting, accessing and translating the evidence to decision-makers, key influencers, and their networks; via customized, accessible products with tailored messaging that avoids jargon; at the appropriate time. Understanding audiences through segmentation (where you divide a large group of users into smaller sub-groups of users (known as segments) based on some type of shared characteristics), as well as providing multiple modalities for access to evidence (such as printed documents, online platforms and in-person events) were highlighted. The literature focuses on being persistent with regular communication, as well as maintaining flexibility in determining the best approach and means to communicate.

Insight 4: Build the capacity of individuals to understand and see the value of evidence. Awareness-raising interventions alone have little evidence of success in improving the use of evidence, but building the skills and motivation to use evidence showed promise. In particular, efforts to build positive attitudes towards evidence use are highlighted as an important complement to awareness-raising work. Marketing the benefits of using evidence in work was also emphasized. This includes social marketing, professional recognition, and norm setting which have been shown to improve evidence uptake.

Yes, these more collaborative, user-centered, and intentional approaches can take more time. But implementing an approach that doesn’t have an evidence base or designing programs without reference to the evidence base is potentially far more costly to the U.S. Government and program participants in the long-term.

How have you integrated evidence into your decision-making processes? What barriers have you faced? And what approaches have worked in your organization?