Welcome to the e-version of the USAID Evaluation Toolkit!
The Evaluation Toolkit:
- Curates the latest USAID guidance, tools, and templates for initiating, planning, managing, and learning from evaluations primarily for USAID staff members who are involved in any phase of the evaluation process.
- Is a resource for USAID staff members and external contractors who participate in or conduct evaluations for USAID.
How to Use this E-Toolkit:
The Toolkit is organized according to the USAID Program Cycle and the phases of an evaluation.
Section 1: Evaluation at USAID (overview of evaluation and the policy context for evaluation at USAID)
Section 2: Evaluation Throughout the Program Cycle (when it is required or encouraged to plan, use, or report on evaluations)
Sections 3 through 5: Phases of an Individual Evaluation
- Section 3: Planning (from deciding to evaluate to procuring an evaluation)
- Section 4: Managing an Evaluation
- Section 5: Sharing, Reporting, Using, and Learning from an Evaluation
- Brief narrative introduces the general requirements and important considerations.
- Sub-thematic areas are listed on the left-hand side and go more in-depth into specific areas or processes.
- Core resources (1) provide further guidance on specific requirements and processes; (2) describe best practices; and (3) offer templates and other tools.
- Additional links that provide easy access to USAID reference documents, reports, and webinars that go into specific evaluation issues in greater depth, and non-USAID resources that may be useful during the evaluation life cycle.
A few resources and additional links are available only to USAID staff. These are indicated by a designation of USAID only.
This Toolkit was developed by the Bureau for Policy, Planning, and Learning Office of Learning, Evaluation, and Research (PPL/LER). Many USAID and USAID contractor staff—from the field and Washington—provided content, comments, feedback, and insights into this Evaluation Toolkit. Their contributions have been and continue to be essential to the ongoing development of this Toolkit.
1. Evaluation Policy at USAID
Evaluation at USAID is defined as the systematic collection and analysis of information about the characteristics and outcomes of programs and projects as a basis for judgments to improve effectiveness, and/or to inform decisions about current and future programming. Evaluation is distinct from assessment (which may be designed to examine country or sector context to inform project design) or as an informal review of projects. It is also distinct from performance monitoring, which is an ongoing process that indicates whether desired results are occurring and whether Development Objectives (DOs) and project outcomes are on track.
The purpose of evaluations at USAID is for learning to improve effectiveness and to ensure accountability to stakeholders. Evaluations may be undertaken at any level of a Mission's or Washington Operating Unit's (OU) portfolio—from an individual award, to a project, to a DO. They may also occur at any time in the life of an individual award, project, or DO.
As noted in the USAID Evaluation Policy, evaluations at USAID will be:
- Integrated into the design of strategies, projects, and activities. Identifying key evaluation questions at the outset will both improve the quality of strategy development and project design and guide data collection during implementation.
- Unbiased in measurement and reporting. Evaluations will be undertaken so that they are not subject to the perception or reality of biased measurement or reporting due to conflicts of interest or other factors.
- Relevant. Evaluations will address the most important and relevant questions about activity, project, and program performance.
- Based on best methods. Evaluations will use methods that generate the highest quality and most credible evidence that corresponds to the questions being asked, taking into consideration time, budget, and other practical considerations.
- Oriented toward reinforcing local capacity. The conduct of evaluations will be consistent with institutional aims of capacity building and respectful engagement with all partners.
- Transparent. Findings from evaluations will be shared as widely as possible, with a commitment to full and active disclosure.
Three documents provide the foundation of all USAID guidance on evaluation at USAID: (1) the USAID Evaluation Policy; (2) USAID Automated Directives System (ADS) 203; and (3) the Standardized Mission Order on Evaluation.
- Reference: M&E POCs List. USAID only.
- Reference: USAID Evaluation Policy: Year One
- Reference: Evaluation at USAID - November 2013 Update
2. Evaluation Throughout the Program Cycle
The USAID Program Cycle is a common set of processes intended to achieve more effective development interventions and to maximize impacts. These processes are defined around a cycle that includes Agency Policy and Strategies, Country Development Cooperation Strategies (CDCS), Project Design and Implementation, Monitoring and Evaluation (M&E), Learning and Adapting, and Budget and Resources.
Evaluations may be planned, conducted, or utilized at any stage in the Program Cycle. This section addresses the various, formal stages of the Program Cycle at which Missions or Washington OUs are required or encouraged to consider whether it would be appropriate to plan for, conduct, or learn from an evaluation.
Evaluation in CDCS
Evaluation in Country Development Cooperation Strategies
A Country Development Cooperate Strategy (CDCS) articulates country-specific development hypotheses and sets forth the goal, objectives, results, indicators, and resources levels that guide Project Design and Implementation, Evaluation, and Performance Management, and inform annual planning and reporting processes.
Evaluations, along with research and other analyses, should be used to inform various sections in the CDC, including the Development Context, Challenges and Opportunities, the Development Hypothesis, and the Results Framework. Per ADS 184.108.40.206, Missions must also explicitly address evaluation in the CDCS. The CDCS must identify:
- High-priority evaluation questions for each DO; and
- At least one opportunity for impact evaluation of a project or project component within each DO.
In addition, Mission’s must incorporate USAID’s Gender Equality/Female Empowerment Policy (ADS 205) by asking relevant questions about whether reducing gaps between males and females contributes to project outcomes.
Evaluation in PMPs
Evaluation in Performance Management Plans
The Performance Management Plan (PMP) is a tool used by a USAID Mission or Bureau/Independent Office (B/IO) to plan and manage the process of monitoring, evaluating, and reporting progress toward achieving the various levels of the approved CDCS results framework (ADS 200.6). The Mission PMP is developed within six months after CDCS approval and is updated over the life of the CDCS.
Per ADS 220.127.116.11, Missions must include an evaluation plan in their Mission PMP to identify and track evaluations across the Mission and over the entire CDCS timeframe. Evaluation plans should include (at minimum): (1) the activity/project/program to be evaluated; (2) evaluation type; (3) possible evaluation questions; (4) estimated budget; (5) planned evaluation start date; and (6) estimated evaluation completion date.
Evaluation in Project M&E Plans
A project is a set of executed interventions, over an established timeline and budget intended to achieve a discrete development result (i.e. the project purpose) through resolving an associated problem. A project is explicitly linked to the CDCS Results Framework.
Project designs should be derived from well-documented, rigorous analysis, including evaluations. In addition, per ADS 201 and ADS 203, Project Appraisal Documents (PADs) must include a Project M&E Plan that includes:
- A description of what type of evaluation, if any, is required;
- A limited number of key evaluation questions that are explicitly linked to specific decisions or essential elements of project learning, along with discussion of proposed evaluation method(s) to answer these questions; and
- A description of any planned evaluations, including estimated evaluation budget and key evaluation tasks and next steps.
Evaluation in Activity M&E Plans
An activity is a sub-component of a project that contributes to a project purpose. It typically refers to an award (such as a contract or cooperative agreement), or a component of a project such as policy dialogue that may be undertaken directly by USAID staff. In the case of an awarded activity, implementers are expected to submit an Activity M&E plan to their Agreement Officer’s Representative/Contracting Officer’s Representative (AOR/COR) within the first 90 days of an award (ADS 203.3.5).
The Activity M&E Plan should include any plans for internal evaluations that are to be undertaken by the implementer. The Activity M&E Plan should also include information for ensuring that any planned external or USAID-led evaluations will have access to appropriate data collected by the implementer, such as performance monitoring data.
In the case of a Government to Government (G2G) activity, the Activity M&E Plan and Learning Plan will generally be created by the G2G POC in cooperation with the host government.
Evaluation in the Budget Cycle
Per the Evaluation Policy and ADS 203, USAID Missions and Washington Operating Units (OU) should devote approximately 3 percent of total program funding on average to external evaluation. In addition, ADS 203 recommends that 5–10 percent of total program resources should be allocated for both monitoring and evaluation (including the required 3 percent of program funds for external evaluation). This does not mean that every project or activity should be evaluated or that 3 percent of the budget of every project, activity or implementing mechanism be set aside for evaluation. The actual costs of M&E may vary depending on the operating environment and the specific types of evaluations the Mission or Washington OU plans to undertake. This recommendation of allocating 5–10 percent of program resources for M&E was established to ensure Missions and Washington OUs dedicate adequate resources to support effective performance M&E.
Per the Standardized Mission Order on Evaluation, the Program Office will calculate on an annual basis a budget estimate for the external evaluations to be undertaken during the following fiscal year. This estimate does not include implementing partners’ internal M&E operations. Most likely, this exercise is best done during the Operational Plan (OP) preparation time. As many external evaluations will also be implementing mechanisms, much of the needed information will need to be prepared as part of the OP.
Evaluation in Portfolio Reviews
A Portfolio Review is a periodic review of all aspects of a USAID Mission or Bureau/Independent Office (B/IO)’s Development Objective, projects, and activities, often held prior to preparing the Performance Plan and Report. Missions must conduct at least one portfolio review per year geared toward strategic review focused on the higher levels of the Results Framework.
Per ADS 203, Missions should consider what has been learned during evaluations (along with other sources of evidence) as a part of their strategic Portfolio Review. In addition, the status of action plans for evaluation findings and their use in respective decisions will be discussed and documented. Finally, Portfolio Reviews (along with Project Reviews and Activity Reviews) should include an update to evaluation plans.
3. Planning an Evaluation
Section 2 noted the various stages of the program cycle where Missions or Washington OUs should formally consider evaluation needs and requirements, including Mission-wide evaluation planning. This section addresses the planning phase for an individual evaluation, from the decision to evaluate to the procurement of evaluation services.
Ideally, evaluation planning should start during the project or activity design stage. This will help ensure that a project or activity and its monitoring system are designed with the planned evaluation in mind. However, the decision to evaluate an activity, project, or program may occur at any time in the program cycle as new evaluation needs are recognized. In addition, evaluations should be timed so that their findings can inform decision making (for example, exercising option years, designing a follow-on project, making mid-course corrections, creating a country or sector strategic plan, or making a policy decision). For a typical performance evaluation, this means the process to solicit an evaluation should begin at least 12–18 months in advance of a decision point.
While early planning is beneficial for all evaluations, it is particularly important for impact evaluations. These studies parallel the life of a project or activity and sometimes require substantial modifications to the design of interventions (e.g. randomized assignment of treatment and control groups, modifications to selection criteria, modifications to roll-out timing, etc.). Understanding impact evaluation requirements at an early stage can help inform the drafting of implementing partner agreements in a way that builds implementer/evaluator cooperation and communicates how the evaluation will affect implementation.
In planning an individual evaluation, sufficient time should be allocated to:
- Draft a strong Statement of Work (SOW) that is peer reviewed prior to finalizing;
- Develop an Independent Government Cost Estimate (IGCE);
- Commission the evaluation to allow partners several weeks to prepare and respond;
- Review proposals and select a finalist;
- Award the contract;
- Conduct the evaluation using high-quality methods; and
- Review, reflect upon, and act on the evaluation findings, conclusions, and recommendations.
For Missions and Washington OUs with M&E Support contracts, the steps outlined here may be somewhat different because the contract to conduct the evaluation may have already been awarded.
Deciding to Evaluate
The decision to evaluate an activity, project, or program should be based on the decision-making needs of a Mission or office, the policy requirements for evaluation, learning needs, and practical considerations.
Certain projects are required to be evaluated:
- Large projects; and
- Pilot activities to be scaled up.
In these cases, decisions need to be made about when and how these projects will be evaluated.
Programs/projects/activities that are not required to be evaluated may still be evaluated to address learning needs or knowledge gaps. In this case, decisions need to be made about whether to evaluate, what type of evaluation (performance or impact) to conduct, what type of evaluation team (internal or external) would be appropriate, and when the evaluation should be conducted. In the case of potentially large, expensive, or lengthy evaluations (particularly impact evaluations), an evaluability assessment may be a worthwhile investment prior to planning an evaluation.
Engaging with Stakeholders
Collaboration is a principle that is integral to all stages of the USAID Program Cycle, including evaluation. For an evaluation to successfully contribute to USAID development results, the Evaluation Point of Contact (POC) and other USAID staff involved in the evaluation must productively engage and collaborate with key stakeholders in the evaluation—USAID staff across various offices, implementing partners, host country officials, project beneficiaries, etc. These stakeholders may contribute to the planning and implementation of the evaluation, serve as primary or secondary audiences for evaluation products, and/or serve as critical actors in ensuring that evaluation evidence is utilized effectively.
Consequently, it is prudent to start identifying and engaging with key stakeholders as early as possible in the evaluation process. A stakeholder analysis is one simple way to start the process of identifying stakeholders and determining how to best collaborate with them throughout the evaluation. At minimum, Project managers and AOR/CORs are responsible for ensuring that implementing partners (IPs) of the activity or project that will be evaluated are aware of any planned evaluations and the steps IPs need to take to ensure a successful evaluation.
Similarly, early dissemination planning for evaluation is critical. As noted in the USAID Evaluation Policy, evaluations of all types will include a dissemination plan. Such dissemination plans can help ensure that appropriate evaluation products are planned and developed to meet stakeholder needs and fulfill USAID’s commitment to transparency, accountability, and learning.
Determining Evaluation Purpose and Evaluation Questions
As early as possible in the evaluation planning phase, the Mission or Washington Operating Unit needs to consider the purpose and audience of the evaluation and the key questions that the evaluation will address. Ideally, the evaluation purpose and questions will be developed during the design of the project or activity to be evaluated. The process of developing the evaluation questions may even inform the decision to evaluate or not.
Evaluation questions are typically developed by the Development Objective team or Technical Office managing the activity, project, or program being evaluated in coordination with the Program Office, which will manage the evaluation in most cases. However, evaluation questions may be based on input from Washington offices, Mission leadership, or other stakeholders, such as implementing partners and host governments. Adequate consultation is essential when defining the evaluation purpose and evaluation questions to ensure that evaluation findings will be credible, relevant, and actionable for decision-makers.
Developing an Evaluation SOW
The development of an Evaluation SOW is one of the most significant steps in the evaluation planning process. The SOW communicates to the evaluation team why the evaluation is needed, how it will be used, and what evaluation questions managers need answers to. Before finalizing the Evaluation SOW, the Mission or Washington Operating Unit Program Office will organize an in-house peer technical review of the Evaluation SOW including no less than two individuals in addition to the Program Office Evaluation POC (or designee). Relevant and non-procurement sensitive parts of the SOW may also be shared with external stakeholders as needed and appropriate. The PO is responsible for ensuring that the SOW is compliant with USAID evaluation policy. Most of the guidance in this section assumes that USAID is the author of the SOW for a competitively procured external performance evaluation. However, SOWs may differ for internal evaluations, for evaluations planned within an existing evaluation contract, or for more complicated impact evaluations.
Developing an Evaluation IGCE
The Evaluation Independent Government Cost Estimate (IGCE), is USAID’s estimate of the costs that an evaluation contractor may incur in performing the evaluation. As with all IGCE’s it serves as the basis for reserving funds during acquisition planning; it provides the basis for comparing costs or prices proposed by offerors/applicants; and it serves as an objective basis for determining price reasonableness in cases in which one Offeror/Applicant responds to a solicitation.
The Evaluation IGCE should be developed concurrently with the Evaluation SOW and should be available to those involved in the peer review of the SOW. The IGCE for an evaluation should follow directly from the information included in the SOW. For instance, the number and complexity of questions, along with the proposed data collection and analysis methods in the methodology section and the team composition requirements in the Evaluation SOW should all be reflected in the IGCE.
- Guidance and Tool: Independent Government Cost Estimate (IGCE) Guide and Template.
- Guidance and Tool: Independent Government Cost Estimate (IGCE) Guide and Template in Excel. USAID Only
Commissioning an Evaluation
For external evaluations, the completed SOW is typically incorporated into a request for proposals (RFPs) or request for procurement. For Missions and Washington Operating Units with existing M&E support contracts, or M&E “Platforms” this will not be the case.
Part of the value-added function of the Program Office is to suggest potential implementing mechanisms for carrying out the evaluation. There are numerous options for procuring evaluation services, including field and Washington mechanisms supported by a variety of USAID offices. Missions should also consider the use of local evaluation contractors. If USAID staff is expected to participate on the evaluation team, that expectation should be acknowledged in the solicitation.
Although procurement typically occurs at the end of the planning phase of the evaluation process, the Evaluation POC (or designee) should work with their contract officers to consider possible mechanisms early in the planning process, since the choice of mechanism can have implications on the budget and timing of the evaluation.
Special considerations should be taken with regard to timing of contracts for impact evaluations. According to the Evaluation Policy, “in cases where impact evaluations are undertaken…a parallel contractual or grant agreement will be established at the inception to accompany implementation.” If possible, the evaluation team should be in place before implementation starts in order to conduct the baseline and provide guidance related to the selection of treatment and comparison/control groups.
- Reference: ADS 300: Agency Acquisition and Assistance (A&A) Planning.
- Reference: Monitoring and Evaluation Platforms: Considerations for Design and Implementation Based on a Survey of Current Practices.
- Tool: Interactive Map of VOPEs (Voluntary Organizations of Professional Evaluators).
- Tech Note: Using PADs to develop RFPs.
- Reference: Impact Evaluation and Activity Implementation Timeline.
- M&E Mechanisms (Field and Washington) USAID Only
4. Managing an Evaluation
This section addresses the management phase of an individual evaluation, from the period following the award of an evaluation contract to the submission of the final report. Following the award of an evaluation contract, the COR (for external evaluations) or the Evaluation Manager (for internal evaluations) serves as the main communication link between USAID and the evaluation team. In most cases, the evaluation will be managed by the Program Office (i.e., Evaluation COR/Manager is a PO staff member). The Evaluation COR/Manager will ensure that:
- The evaluation team’s final evaluation design meets the Agency’s needs;
- The evaluation team has access to the necessary information (e.g. project or activity reports, performance monitoring data, key contact information, etc.);
- The evaluation team is proceeding with the evaluation as envisioned;
- Coordination between evaluators and implementers is smooth; and
- A final report is reviewed, approved, and disseminated.
From Award to Approval
In the period following the award of the evaluation contract, but prior to data collection, an evaluation design is the one deliverable required by USAID policy. Other deliverables also may be due during this period, depending on what was requested in the Evaluation SOW.
An evaluation design describes and documents how the data collection and analysis methods will be used to produce credible evidence for answering all of the evaluation questions within the time and budget constraints. Clear articulation of the evaluation design aids USAID and other stakeholders in discussing these choices with the evaluation team, but the level of detail in an evaluation design may vary depending on the complexity of the evaluation, overall level of effort, and other factors. An Evaluation Design Matrix is a standard tool for outlining the components of an evaluation design and is highly recommended for use by evaluation teams.
Per the Evaluation Policy, “Except in unusual circumstances, the design [of the evaluation] will be shared with country-level stakeholders as well as with the implementing partners for comment before being finalized.”
Additional deliverables at during this period typically include:
- A workplan that describes the schedule, activities, and milestones of the evaluation team;
- An inception report or background report that addresses what the evaluation team has learned based on program documents provided to them;
- An in-brief or series of in-briefs, either in person or virtual; and
- Other possible deliverables, such as an evaluability assessment.
During this design period, the Evaluation COR/Manager should consider the possibility of revising evaluation questions based on evaluation team input. Any revisions to the questions in the Statement or Work should be documented in writing in the evaluation report. The Evaluation COR/Manager should also consider if the design complies with ethical standards for protection of human subjects.
Conducting an Evaluation
USAID staff members typically manage evaluations on behalf of USAID, while the design and conduct of an evaluation is more typically the responsibility of an externally contracted evaluation team. USAID staff may participate in these external evaluations, provided that the team leader is an externally contracted evaluator with no fiduciary relationship with the implementing partner. In addition, USAID staff may lead and/or participate in internal evaluations.
Whether leading an evaluation, managing an evaluation, participating on an evaluation team, or just reviewing an evaluation report, it is beneficial to be familiar with the typical designs and methodologies of USAID evaluations. The field of program evaluation is quite diverse, and numerous books, journals, and websites are dedicated to describing the various approaches, models, designs, methods, techniques, and practices in conducting program evaluations. This section provides some limited guidance on designs and methods for conducting an evaluation for USAID.
The USAID Evaluation Policy places considerable emphasis on high-quality evaluation methods. It notes:
“Evaluations will use methods that generate the highest quality and most credible evidence that corresponds to the questions being asked, taking into consideration time, budget, and other practical considerations. Given the nature of development activities, both qualitative and quantitative methods yield valuable findings, and a combination of both often is optimal.... No single method will be privileged over others; rather, the selection of method or methods for a particular evaluation should principally consider the empirical strength of study design as well as the feasibility.”
- Guidance: The Road to Results: Designing and Conducting Effective Development Evaluations.
- Guidance: Impact Evaluation in Practice.
- Guidance: Real World Evaluation: Working Under Budget, Time, Data, and Political Constraints: A Condensed Summary Overview.
Managing and Monitoring an Evaluation Team
The responsibilities of an Evaluation COR/Manager for an evaluation contract or task order are technically no different than the responsibilities of a COR for any other implementing mechanism. In practice, though, managing the implementation of an evaluation differs in significant ways from managing the implementation of other development activities.
Evaluations typically have much more compressed timelines compared to other activities, requiring quick responses and adaptations when problems arise.
Evaluations also rely heavily on the cooperation of other USAID partners. The Evaluation COR/Manager must help mediate and manage the relationship between the evaluation team and the implementing partner being evaluated. This relationship can place a considerable burden on the implementing partner as they assist the evaluation team in obtaining documents, participate in interviews, and facilitate access to beneficiaries. This is particularly true for experimental designs in impact evaluations, which require that the implementing partner adhere to pre-specified treatment and control groups.
Finally, managing an external evaluation team requires the Evaluation COR/Manager to carefully balance USAID’s involvement to ensure a high-quality, useful, and on-time product with the need to protect the independence of the evaluators and the evaluation report.
From Draft to Final Report
USAID’s Evaluation Policy and ADS 203 set requirements for evaluation report structure and content. The report must present a well-researched, thoughtful, and organized effort to objectively evaluate a USAID activity, project, or program. Findings, conclusions, and recommendations must be based in evidence derived from the best methods available given the evaluation questions and resources available. The evaluation methods, limitations, and information sources must be documented, including by providing data collection tools and the original Evaluation SOW as annexes to the main report.
Following the submission of the draft evaluation report, the Mission or Washington Operating Unit Program Office will organize an in-house peer technical review to assess the quality of the draft report and ensure that comments are provided to evaluation teams. The PO will be responsible for ensuring that the final report is compliant with USAID evaluation policy and ADS 203.
The draft report will be also be shared with implementing partners whose projects (or activities) are examined in the evaluation and other organizations who contributed funding to the evaluation or the project/activity being evaluated. Funders, implementers, and members of the evaluation team are to be provided with the opportunity to write a “statement of differences” to address any unresolved differences of opinion to be appended to the final evaluation report.
- Example: Sample Evaluation Covers.
- Guidance: USAID Graphic Standards Manual.
- Tool: Checklist for reviewing a randomized controlled trial.
5. Sharing, Reporting, Using, and Learning
Completing an evaluation report is not the end of the evaluation process. As the evaluation report moves toward completion, the Mission or Washington OU that commissioned the evaluation enters into the key phase of sharing, reporting, using, and learning from the evaluation.
Sharing: Transparency is a key practice of evaluation at USAID. As noted in USAID’s Evaluation Policy, “Findings from evaluations will be shared as widely as possible, with a commitment to full and active disclosure." At minimum, this requires the posting of evaluation reports to the Development Experience Clearinghouse (DEC) and evaluation data to the Data Development Library (DDL).
Reporting: Planned, ongoing, and completed evaluations must be reported annually in the Evaluation Registry of the annual Performance Plan and Report.
Using and Learning: Given the time and effort that is expended in planning and conducting evaluations, it is essential that Mission and Washington OUs use evaluations to understand performance, test development hypotheses, question assumptions and cause - and - effect relationships, and, ultimately, manage for results and learning. At minimum, following the completion of an evaluation, Missions and Washington OUs should respond to the evaluation through the development of an action plan for addressing the evaluation findings, conclusions, and recommendations
Missions and Washington Operating Units (OU) should share and openly discuss evaluation findings, conclusions, and recommendations with relevant partners, donors, and stakeholders, unless there are unusual and compelling reasons not to do so.
Missions and Washington OUs should revisit their evaluation stakeholder analysis and dissemination plan toward the conclusion of an evaluation to ensure that it still reflects the priorities for dissemination. While sharing the evaluation report is the most typical form of dissemination, Missions and Washington OUs should also consider other methods of dissemination, such as hosting briefings with local stakeholders, partners, and other donors to discuss evaluation findings; featuring evaluation findings on their website, such as through articles or blog posts; and holding press conferences and issuing press releases.
In many cases, USAID Missions should arrange the translation of the executive summary into the local written language.
Two forms of evaluation report sharing are required:
- First, the Program Office must ensure that the final evaluation report is posted on the Development Experience Clearinghouse (DEC) no later than three months after completion. Exceptions to this requirement are granted only in very rare circumstances (see Guidance on Exemptions to Public Disclosure of USAID-funded Evaluations).
- Second, in addition to posting the evaluation report to the DEC, Program Offices must post quantitative data from the evaluation to the Data Development Library.
The annual Performance Plan and Report (PPR) documents USG foreign assistance results achieved over the past fiscal year and sets targets on designated performance indicators for the next two fiscal years. The PPR also includes the Evaluation Registry as a sub-module for documenting each Mission and Washington Operating Unit’s work on evaluation.
The Evaluation Registry includes information on evaluations completed in the most recent fiscal year, evaluations that are currently ongoing, and evaluations planned for the current year and two additional out-years. This includes required, non-required, external and internal evaluations. As of FY2013, the data from the Evaluation Registry are also used to calculate the targets and actuals for the USAID Forward Evaluation Indicator. Evaluation status and budget data from the Evaluation Registry is critical to helping USAID understand the number of evaluations completed across the Agency, the totality of budget resources being devoted to evaluation, and trends over the fiscal years. These data also help USAID demonstrate to external stakeholders, such as the White House’s Office of Management and Budget, the priority that USAID places on evaluation.
- Reference: Performance Plan and Report Guidance (FY2015). USAID only.
Utilization and Learning
The value of an evaluation is in its use. Evaluations should inform decision-making, contribute to learning, and help improve the quality of development programs. At minimum, USAID Program Offices should lead Missions and Washington Operating Units through the process of:
- reviewing findings, conclusions, and recommendations of evaluations that relate to their activities, projects, and DOs;
- identifying any management or program actions needed; and
- assigning responsibility and time lines for completion for each set of actions.
While learning and utilization is most often considered at the conclusion of the evaluation process, learning and utilization can happen at various phases in the evaluation process and stages of the program cycle. Evaluation use and learning may occur before or during the evaluation, shortly after it is completed, or long after the findings have been presented. It may occur during the development of a CDCS, the design of project, portfolio review. Utilization and learning should be planned for and actively facilitated, whenever evaluation use occurs.
Assessing the Evaluation Process and Evaluator Performance
Following the completion of the evaluation, the Evaluation COR/Manager and others involved in the evaluation should consider not just the content of the evaluation report, but what they have learned from the entire evaluation process that might be help in conducting the next evaluation. An After Action Review (AAR) is one formal means of capturing such lessons.
If the evaluation was contracted, the Evaluation COR should, when applicable, access the Contractor Performance Assessment Reporting System (CPARS) to file a Contractor Performance Assessment Report within 60 days of the completed evaluation. CORs completing an assessment report should ensure that the report contains an accurate portrayal of the contractor’s performance. Contractors utilize the completed past performance reports when responding to solicitations. These reports are also used by Contract Officers and CORs when assessing the past performance of contractors and to incentivize contractors to produce superior products and services.
- Guidance: After Action Reviews (AAR)
- Guidance: User Manual for Contractor Performance Assessment Reporting System (CPARS).