A New Normal: Adapting our Approach to MERLA during COVID-19
If you’d asked what we thought would be a “worst case scenario” for our work in monitoring, evaluation, research, learning & adapting (MERLA) before March of this year, we might have answered “data quality checks are not going well” or “it’s been tough to get a Learning Session scheduled for the whole team.” However, in those scenarios, we are usually able to uphold our assumptions to continue our work.
COVID-19 has provided the unique opportunity for our MERLA team at RTI International to quickly learn and adapt as we work through the short- and long-term challenges of implementing projects at a time like this. Data for decision making, and a pretty level head, are critical factors when everything is uncharted territory. In any organization during “normal” times, knowledge management (KM) can pose a huge challenge and different initiatives to improve processes can only get us so far. We need people to earnestly participate and provide the information to oil our KM machines. As we all puzzle through how to learn and adapt to the new challenges that we face, it’s important that we create opportunities to learn from one another and that we learn to ask for help.
Our MERLA Community of Practice, a group of more than 70 staff at RTI, has worked to bring staff together to learn from one another, problem-solve, and fill gaps in our knowledge over the last few months. In addition to our external monthly Learning and Adapting during COVID-19 webinar series, where we’ve invited partners and collaborators from various organizations (USAID, Harvard University, UNICEF, World Bank, and others) to share their lessons learned during the pandemic, we’ve been organizing internal avenues for knowledge sharing across our organization. We recently hosted a “MERLA during COVID-19 Forum,” where projects from across our different technical areas presented what they’re seeing on the ground and how they are learning and adapting their approaches to monitoring, evaluation and learning (MEL) and KM during this challenging time.
Here are several key lessons learned from our projects as they adapted their MERLA methods during COVID-19:
1. Adapt questionnaires for phone surveys and use tools and applications available to respondents. Instead of attempting to convert a traditional face-to-face questionnaire to be conducted over the phone, customize the survey conducted over the phone using SMS, CATI, IVR, WhatsApp, Zoom, or other applications that are available to respondents. This could include shortening the overall survey time to account for noise or distractions, providing an incentive like a phone credit, and establishing trust by familiarizing respondents with the process before the actual survey call. In a recent webinar, RTI hosted a panel of experts working on phone surveys to collect data for COVID-19 in low and middle income countries.
2. Coordinate and collaborate with MEL staff and focal points in government institutions. Coordinating and collaborating among MEL staff, technical staff, and government focal points at the central and regional levels is necessary to effectively collect, centralize, and quality check data from different institutions. For instance, when respondents were unreachable or did not respond on time, the USAID Wildlife Asia activity tapped into their network of local partners and technical experts to reach out to participants.
3. Adapt trainings, data collection tools, and applications to the local context. Planning for the unique circumstances of phone interviews can create a better experience for enumerators and respondents. For example, the USAID Tusome Pamoja activity in Tanzania provided enumerators with clear guidelines and tips to help them adapt to remote sampling, interviews, and data collection. Some of the key success factors included developing flexible data collection protocols, setting up WhatsApp groups among enumerators so they could learn from each other, collaborating and adapting in real time when faced with difficulties, and preparing replacement respondents in case of connectivity issues.
4. Flexibility and follow-up are critical to sustaining relationships. Developing compatible protocols and instruments to the remote setup is essential, while flexibility and follow-up contribute to later success. For example, the USAID/Governance for Local Development (GOLD) activity in Senegal has reported that using video conferences to train partners to submit their data through an online data collection system has proven useful in maintaining data flow. They created a WhatsApp group to communicate to partners, are following up via telephone calls for feedback and data quality reviews and collaborating with other programs to share data and lessons learned.
As we continue to share knowledge within RTI and with the development community, we’ve developed a few general observations. First, learning from each other’s experience, good practices and lessons is critically important, especially in an extremely vulnerable and stressful time. The MERLA during COVID-19 Forum provided an important opportunity for project teams to ask each other questions about methodologies, tools and technologies that have been working, about how to problem-solve tough decisions while ensuring reliable data is collected on time, and most importantly, to ask “how are you adapting?” Second, there’s no one size fits all solution for adapting to this new environment. Each country, project, and person have their own unique circumstances that are important to acknowledge and to understand while we all navigate this new terrain. Third, be open to new ideas, because if there was ever a time to test what we thought we knew about anything, the time is now. Let’s embrace that some of the best ideas come when we’re tested. And last but certainly not least, let’s share with each other, across our teams, organizations, and this international community what works and what doesn’t, because it’s a long road ahead and remember, we’re all in this together.