1. Where does complexity-aware monitoring fit in USAID’s Program Cycle?
Complexity-aware monitoring (C-AM) is a responsive approach, and it can be implemented at several points in the Program Cycle. We envision that it will be particularly helpful to think about C-AM at the project design and monitoring and evaluation (M&E) planning and updating phases, at the completion of a Country Development Cooperation Strategy (CDCS) process, or if things are changing and you’re in need of adaptive management (the “learning and adapting” phase of the Program Cycle). These times are all appropriate for C-AM implementation, although no time is prohibitive.
2. How can we incorporate learning into adaptive management given USAID's procurement rigidity? What's the link with C-AM?
This is a really important question, and we'd like to correct a common misunderstanding: the procurement system is probably not as rigid as you may think. In our experience, contracts officers vary; but we're often very impressed with how forward thinking USAID's contracts officers are, particularly when the program's goals and expectations are clearly explained. If you discuss your needs openly, chances are there's a solution available to you. Adaptive management requires flexibility, and C-AM helps to provide the timely monitoring data to help inform this flexibility.
USAID employees can find additional guidance on Programnet, including a brief that identifies a few procurement mechanisms which are inherently more flexible options. USAID staff can download the brief here.
3. Can complexity-aware monitoring be incorporated into the usual USAID trainings (Programming Foreign Assistance, Project Design and Management, Evaluation), where appropriate?
As we learn more during our trials and start to build a body of evidence, we will incorporate our findings into relevant trainings.
1. Which projects were selected for the complexity-aware monitoring trials?
First and foremost, we looked for partners who have identified a monitoring need related to a complex aspect of their project. A project may have been a good candidate if performance monitoring wasn't meeting information needs because of any of the following:
- Cause and effect relationships are poorly understood, thereby making it difficult to identify solutions and draft detailed implementation plans in advance
- Expected results require revision to take advantage of new needs and opportunities
- Adaptive management is necessary to steer effectively in a dynamic context
- The purpose of the project is to influence social change or create a new solution to an intractable problem
2. What is the ideal time to consider applying complexity-aware monitoring in a project—in the design stage or during implementation?
Complexity-aware monitoring can be implemented at any point in the Program Cycle. However, for the purposes of these trials, we are looking for projects that are already operational and that have a current monitoring need. For those of you who are still at the project design stage, we would love to talk about incorporating complexity-aware monitoring into your project or activity M&E plan. After the project is running, you may then be an ideal partner for us in a later phase of the trials.
3. Where can I find examples of complexity-aware monitoring in USAID?
Most of the promising approaches on trial are backed by a significant body of theory and practice, but they had not previously been used to monitor USAID strategies and projects. The purpose of these trials was to generate a body of knowledge about the benefits of these approaches, and what supports their use, and under what conditions.
4. Why trials? Why not just provide manuals on new monitoring approaches?
When we talked to M&E innovators in the Agency, we found that they struggled with some common challenges including:
- Selecting an appropriate M&E approach to meet their information needs and suit the project and its context
- Understanding which adjustments to the M&E approach facilitate use and which ones compromise data in the application of the M&E approach
The trials were designed to help innovators address these challenges, plus capture the learning so that others can also benefit.
5. What is a typical timeline for a trial? How long did the trials take?
The trials were designed to cover the entire monitoring cycle from design to use of data. A trial beganwith a collaborative process to identify a monitoring need and select the most appropriate approach, continues through planning, preparation, training, and actual monitoring, and ended when the data informs decision making. Many partners required support for approximately one year. However, some approaches took less time.
6. What were the time and resource requirements required from trial partners?
Time and resource requirements for each trial depended on the project’s monitoring need and the approach selected to meet that need. The five recommended approaches offered a range of resource thresholds, our goal was to help partners find the right balance between the usefulness of the information with the effort and cost to obtain it. There are very real time and resource (e.g. LOE, funding) constraints on potential partners, and we developed plans that fits within partner's available capacities.
7. What kind of support and technical assistance were provided?
Support to trial partners from the Office of Learning, Evaluation, and Research (LER) began during start-up. The Complexity-Aware M&E Team helped partners identify how monitoring data can support adaptive management of your project in complexity. Then, LER worked with partners to select the most appropriate monitoring approach. LER’s Resource Panel of international experts assisted with the design of the trial. After the trial launched, an expert provided tailored technical assistance on application of the monitoring approach. Each trial convened a Learning Circle that supported the use of the monitoring approach for the purpose of serving the project’s monitoring needs. Other members of the Learning Circle—including implementing partner representatives, USAID staff from the relevant USAID Mission or Operating Unit, members of the LER Office—collaborated to problem-solve and ensured that the trial produces useful data that meets the project’s monitoring needs.