DRG Center & PPL Co-Host Third Annual Impact Evaluation Clinic

Mar 14, 2016 by Jessica Benton Cooney Comments (0)

During the week of February 29-March 4, USAID mission representatives from Peru, Macedonia, Honduras, Guatemala, and El Salvador teamed up with academics, Impact Evaluation (IE) experts, and USAID/Washington staff to design five new IEs during the third annual Impact Evaluation Clinic in Washington, D.C.   

Woman presenting Morgan Holmes, Learning Division, DRG Center presenting on Implementing Impact Evaluations at the Clinic.
Photo Credit: Jessica Benton Cooney. 

"This third Clinic built on positive experiences in Thailand and Tanzania, bringing political science professors together with USAID field officers for an exchange of learning to date and collaboration on future research," said Morgan Holmes, Learning Division, DRG Center. "Holding the Clinic in Washington, D.C., allowed us to add some non-USAID practitioners to the mix. We heard from IE experts from the World Bank and Millennium Challenge Corporation on their experience conducting IEs and were glad to confirm some of our shared findings as well as offered new ideas for how to focus our work going forward."

Man speaking  Darren Hawkins, Brigham Young University, presents at the Clinic on Evidence Review: Information Campaigns, and Rachel Sorey, USAID/Peru.
Photo Credit: Jessica Benton Cooney

The innovative Clinic, co-sponsored by the USAID Policy, Planning and Learning Bureau's Office of Learning Evaluation and Research (PPL/LER) and the DRG Center, provided instruction on the randomized control trial method of IEs, and enabled teams to apply this knowledge to the development of IE designs for new DRG projects. Clinic sessions included insightful evidence reviews from academics, briefings on IE methods and threats, and discussion of real life experiences with IE implementation.

man speaking Victor Merino, USAID/Peru speaking during a panel discussion at the Clinic.
Photo Credit: Jessica Benton Cooney

Participants also learned about IE design, issues such as sampling, power calculations, theory of change, and measurement. "The Macedonia Mission came prepared with a good basic knowledge of the costs and benefits of impact evaluation," said Laura Adams, Learning Division, DRG Center. "Therefore, we were really able to focus in on the key intersection of what the Mission and academic teams both wanted to know about mobilizing civic participation among youth, and what kind of evaluation design might be possible given the practicalities of the program being designed."

participants speakingParticipants during teaching sessions of the Impact Evaluation Clinic.
Photo Credit: Jessica Benton Cooney. 

The Clinic projects, which focused on aspects of civic participation, youth, and environmental justice, will be finalized in partnership with host Missions and academics during subsequent scoping trips. "This year's clinic featured an "evaluation retrospective" by University of Pittsburgh's Steve Finkel, who has been evaluating USAID interventions since 1997," said Aaron Abbarno, Learning Division, DRG Center. "A major takeaway from his remarks is that USAID has become much better at evaluating programs rigorously over the past 20 years. USAID relies on increasingly sophisticated methodologies to make increasingly accurate statements about cause and effect. Impact evaluations complement broader program evaluations; taken together, we can identify and explain wide-ranging consequences of USAID programs." 

This IE Clinic represents an important milestone in the development of the DRG evidence base necessary for telling our story, refining our programming, and developing our staff. For more details about the Clinic, please visit the 2016 Impact Evaluation Clinic website.


Evaluating and Learning from USAID Democracy, Human Rights, and Governance Programming

Apr 16, 2015 by Jessica Benton Cooney Comments (0)

DRG Center Director, Neil Levine presenting on the DRG Strategy, objectives, budget, and general DRG priorities. Photo: Jessica Benton CooneyThe DRG Center and the Policy Planning and Learning Bureau's Office of Learning Evaluation and Research (PPL/LER) organized a week-long Impact Evaluation Learning Clinic in Dar es Salaam, Tanzania from March 23-27. The purpose was to generate rigorous evidence to concretely demonstrate the impact and the difference democracy, human rights, and governance makes in USAID programming.

The Clinic brought together DRG technical staff from Missions and USAID/Washington, as well as leading DRG impact evaluation academic experts.  A total of 38 applications from field missions were submitted for participation in the Clinic, and seven projects were selected based on how promising they were for designing evidence based DRG activities that lend themselves to rigorous impact evaluation.  Participating Missions included: DRC, Haiti, Malawi, Mali, Nepal, Senegal, and West Bank and Gaza. 

"These are activities that are new, untested, and are influential or are activities in subsectors where USAID spends a significant amount of money," said Altin Ilirjani, DRG Center Technical Lead for Learning.

The primary objective of the Clinic was to apply the broader social science learning on program design and create new evidence-based DRG activities that lend themselves to experimental impact evaluation methods. The intention was to develop impact evaluations with generalizable results to USAID learning activities across the over 87 countries where USAID operates.  

"The Clinic was particularly relevant to the DRG strategy, as we integrate DRG practices across development sectors," said DRG Center Director, Neil Levine. "This requires us to be a learning organization, where we are applying back in the lessons learned on an ongoing basis." 

The week was structured around a clinic-based model with workshop and training sessions, led by the academics and USAID Washington staff on topics that included an overview of impact evaluation methods, political economy analysis, local solutions, academic evidence reviews, and political indicators. "I think the DRG Clinic is really successful in that it brings together top notch academics that are well versed in some of the academic studies that interest people working at USAID, and of course USAID Mission Staff," said Matt Winters, University of Illinois at Urbana-Champaign, Assistant Professor of Political Science. Future activity design plans were developed in clinics and presented the last day.

"As a citizen, as well as others in Nepal, we realize that political parties are not very responsive to their constituencies," said Murari Adhikari, USAID/Nepal in regard to how he plans on applying knowledge gained during the Clinic. "They go out for their elections, they win their elections, they forget their promises. So we would like to take these lessons learned from the Clinic and design a very robust impact evaluation to see whether the interventions work."

Follow up on the implementation of the projects will include participants receiving Mission leadership buy in of their projects, after which academics will organize scoping trips to further develop the impact evaluation designs in collaboration with the Missions and implementers. "The DRG Center is going to follow the implementation of the activities throughout the life of the projects," said Altin. "We will also ensure that the knowledge generated will be incorporated into the next generation of DRG foreign assistance activities."

Project Haiti: From left: Alphonse Nkunzimana, USAID/Haiti, Tara Slough, Columbia University, Gerard Fontain, USAID/Haiti, Chris Fariss, Penn State University, Frantzdy Herve, USAID/Haiti, Sarah Logan, DRG Center, GROL Division. Photo Credit: Jessica Benton CooneyThe DRG Center will also work directly with the Missions on the dissemination plan for the knowledge that will be developed through the impact evaluations, and assist in their completion over the next three to five years.  Most of these evaluation projects will be procured through the DRG/LER contract with co-funding from the DRG Center and PPL/LER. 

"The Clinic allowed me to learn and listen to different lessons from different countries," said Mildred Pantouw, USAID/Indonesia. "And how each country has their unique problems as well as opportunities, as well as how important it is to identify and be specific with a problem and apply the type of methodologies and evaluation to support the intervention." 

Subscribe to RSS - jbentoncooney's blog