When Performance Monitoring Stops Performing

This article has been cross-posted from the BEAM Exchange blog. The blog article was written by Heather Britt and Jessica Pomerantz to show how complexity-aware monitoring approaches overcome the three blind spots of performance monitoring.

Has this happened to you?

You're the manager of a tractor access programme for smallholder farmers. Three days after you sign agreements for 100 tractor service providers for your region, floods come and wipe out the crops. Now you've got nothing to show for growth in average hectarage per day.

You're the monitoring and evaluation officer for a market development project. After the first season of farmer trainings are complete, crop sales surge. You and the staff celebrate, and you begin typing up the fantastic results with big dreams of scaling up. Later that day, one of the farmers tells you that sales are up because several neighbouring villages are suffering from crop blight.

You're leading a gender inclusion agricultural project to try to incorporate more smallholder women farmers into local markets. Towards the end of the project, an armed conflict erupts. Revolutionary factions conscript a large number of male villagers—leaving women behind to conduct most of the local crop production and sales, far beyond the targets in your logical framework.

If you have experienced monitoring issues like these, then you've encountered the three blind spots of performance monitoring.

What are the three blind spots of performance monitoring?

Performance management isn't just a suggestion — it's the law for those of us based in and funded by the United States. Around the world, many donors take a similar stance and require performance management, which usually means monitoring indicators.

But typical performance monitoring, based on indicator measurement of results fails to capture three major areas:

• Outcomes outside those desired by the project planners
• Alternative causes of outcomes
• Feedback loops and non-linear pathways of contribution

In predictable situations where particular interventions produce the desired effect every time, performance monitoring works well. But that’s not where most of us are working! Instead, we encounter situations like those described above. Our monitoring tactics need to adapt so that as managers we aren't blind-sided by outcomes outside those desired, alternative causes of outcomes, or a fuller range of non-linear pathways of contribution. We in the international development field need a new approach to monitoring to adapt to the dynamic environments in which we operate. 

What is complexity-aware monitoring?

Complexity-aware monitoring is intended to support adaptive management rather than supply reports on results. It overcomes performance monitoring’s three blind spots to inform decisions in situations of high uncertainty and low agreement. Complexity-aware monitoring helps us understand the relevant interrelationships in a situation, engage with multiple perspectives, and reflect on boundary judgements. Monitoring matches the pace of change rather than pre-determined reporting schedules. But don’t throw away your performance measurements just yet! Complexity-aware monitoring is a complement to existing systems, not a replacement. 

At USAID, we’re focusing on five new approaches (for more information on each method, read the Complexity-Aware Monitoring Discussion Note): 

• Sentinel Indicators. Alert you to changes in the mutually influencing interrelationships between a project and its context.  
• Stakeholder Feedback. Uncover the diverse perspectives of partners, beneficiaries or those excluded from a project through a variety of methods (citizen report cards, a survey, or a public opinion poll).
• Process Monitoring of Impacts. Monitor the processes that are key to the success of the project as well as the context’s influence on those processes.
• Most Significant Change. Capture a broad range of results and make diverse perspectives about those outcomes explicit.  
• Outcome Harvesting. Gather evidence on a broad range of results, then work backwards to determine if and how the project contributed.

What is complexity-aware monitoring going to do for me?

For one, complexity-aware monitoring is going to help you overcome the three blind spots of performance monitoring. If you were the manager of that tractor access programme and you employed outcome harvesting as your monitoring tactic, you would learn how your project contributed to local collaborations and building project management skills. If you were the monitoring and evaluation officer of the market development project, you could have used stakeholder feedback to discover that access to crop insurance is most important to local farmers. If you were leading the gender inclusion agricultural project, most significant change would help you both capture the unexpected results of the project and to understand how various groups (women and men) in the community valued those outcomes.  

Complexity-aware monitoring isn't a typical performance management tool — it will help you learn about your project processes, outcomes, and context in a way that indicators often can’t convey. Giving managers the information they need to navigate complex situations of great uncertainty will see adaptation increase, resources become more impactful, and we can all work more effectively to solve the intractable development problems of our time.  

USAID is currently sponsoring trials to test these new approaches in its projects. The latest resources and news can be found on the Learning Lab. Tell us about your interest and experience with complexity-aware monitoring by completing this short survey. We look forward to hearing from you. 

Heather Britt is the technical lead for USAID's Complexity-Aware Monitoring and Evaluation initiative in the Office of Learning, Evaluation and Research.

Jessica Pomerantz is a Presidential Management Fellow assisting the Complexity-Aware Monitoring and Evaluation initiative.

See more on measuring results in the BEAM Guidance section.

Read the original post and post your comments on the BEAM Exchange blog.