Expanding the Reach of CLA Through Wide Stakeholder Engagement
Jan 2, 2019 by USAID/Jordan, USAID/Jordan Monitoring and Evaluation Support Project (MESP)Comments (0)
The USAID Collaborating, Learning, and Adapting (CLA) Framework and Maturity Tool provides an engaging participatory approach to assist missions in being more intentional about implementing CLA practices. While the CLA Framework and Maturity tool are primarily designed to help USAID Missions plan and implement CLA approaches, the key steps in the implementation of the Maturity tool, including self-assessment, establishing a vision, developing an action plan and tracking progress, are also relevant for a wider set of stakeholders. This provides a great opportunity to extend and apply CLA more broadly in the development arena.
This is particularly relevant as USAID Missions collaborate at multiple levels. Engaging the CLA framework to enhance CLA integration across a broader array of stakeholders has clear advantages. By leveraging stakeholder knowledge, it allows for better coordination and more effective division of labor, an improved understanding of the context and how it may be evolving, as well as the ability to identify key opportunities for adaptive management. Applying the CLA framework among a broader community of stakeholders, however, requires the buy-in of the various stakeholders, and a shared understanding of the framework and how it may support the achievement of their development objectives. Furthering this understanding, in turn, requires making the framework and its key aspects more accessible to a broader audience.
Enhancing CLA Awareness and Utilization in Jordan
Building on a training provided by the USAID Bureau for Policy, Planning, and Learning and LEARN contract teams in 2017, USAID/Jordan’s Monitoring and Evaluation Support Project (MESP) embarked on a multi-staged approach to enhance CLA awareness across a wide array of stakeholders. Primarily, this involved creating an interactive, tailored workshop, introducing a wide range of stakeholders to the concept of CLA, the Maturity Tool, and the Self-Assessment Process. In a further effort to expand awareness and strengthen the utilization of CLA across Jordan, MESP released the first ever Arabic translation of the CLA Maturity Tool.
Not only did MESP strengthen awareness among USAID staff and implementing partners, but also expanded awareness and potential utilization of CLA to a wide range of local stakeholders, including the Government of Jordan and important local civil society organizations. As the introduction of CLA to local stakeholders was a new approach, MESP further tailored the CLA workshop to contextualize CLA for both USAID and non-USAID stakeholders and ensure CLA was accessible to this broader audience. In total, MESP strengthened CLA awareness and interest in utilizing CLA among 68 individuals from 35 organizations (15 implementing partners, 6 local organizations, and 14 Government of Jordan or semi-governmental institutions) across two workshops held on September 17 and October 22, 2018. Government Ministries supported included a range of key Ministries partnering with USAID to ensure sustainability of USAID-funded development initiatives. Key Ministries include: Ministry of Planning and International Cooperation, Ministry of Health, Ministry of Municipal Affairs, Ministry of Social Development, and Ministry of Water and Irrigation, among others.
As a result of the workshops, over a third of participant organizations expressed interest in MESP providing further support on CLA, including facilitation of the CLA Self-Assessment process to strengthen CLA practices. To follow-up and strengthen this demand for CLA, MESP is exploring options to co-facilitate the self-assessment process and/or provide technical assistance as needed with interested institutions and organizations. Ultimately, as CLA awareness and utilization increases, so too is the enabling environment strengthened for achieving USAID’s Development Objectives.
Bringing a Collaborative Approach to Building Resilience in Northern Kenya
Dec 6, 2018 by Dorine GengaComments (0)
This blog is cross-posted from the American Foreign Service Association blog. In addition to submitting a winning case in the 2018 Collaborating, Learning and Adapting Case Competition Dorine Genga is the program management assistant for USAID/ Kenya and East Africa. Ms. Genga’s responsibilities include monitoring, evaluation and learning for the Water, Sanitation and Hygiene program and Feed the Future livestock and resilience activities. This portfolio includes $107 million in USAID investments in Kenya. Prior to joining USAID, Ms. Genga worked with an international nongovernmental organization on refugee resettlement. She holds a bachelor of education degree in business studies and geography.
Following the 2011 drought in Kenya, many families lost their entire herds, leaving them without any source of income. The vendors at the Merille livestock market faced many challenges, including being forced to travel long distances in search of pasture, water and veterinary care. Credit: USAID.
What does drought look like to you? For the pastoralist communities in northern Kenya that rely on livestock herds to sustain their families, a drought can mean the death of their cattle, which may be their sole source of income. In northern Kenya, traditional culture places enormous value on amassing large herds of cattle, camels and goats as a sign of wealth and status in the community. These herds represent the entirety of these people’s assets, for they are generally averse to saving in formal banking systems.
In the north, the small percentage of the community that engages in farming is also threatened by droughts as crops wither and die. Severe droughts cause major challenges for farmers and pastoralists alike, not only in terms of their income, but also their ability to survive.
Drought-induced poverty often results in other challenges, such as malnutrition for children and expectant mothers, conflict, insecurity and increased vulnerability of women and children, because the men leave their homes and trek for longer distances in search of pasture and water for the already emaciated livestock. Sadly, I have seen this firsthand.
In 2011, an 18-month drought affected the Horn of Africa, leaving more than three million Kenyans in urgent need of food aid. In the wake of this devastation, USAID started the Resilience and Economic Growth in the Arid Lands project to spur livelihoods in a zone that was traditionally dependent on humanitarian assistance. Working with communities, the goal was to strengthen families so that they could weather the next shock—whether it be drought, a storm or cattle disease—without completely losing their livelihood.
I participated in a REGAL field visit in 2013, where I listened to pastoralist traders at the Merille livestock market describe the challenges they faced. One of the stories that touched me was that of one man who was robbed on his way home after selling his camels for $3,000. He was devastated. He asked for USAID’s assistance to get mobile banking to the area so that these transactions didn’t force them to carry large amounts of paper currency. A female trader also raised concerns about the long distance she had to travel to get veterinary services when her goats and sheep showed signs of infection. In one of the villages we visited, a lady stated she had only been eating one meal a day and would sometimes go two days without a meal. Her children looked really thin and miserable.
Following this field trip, I returned to the office with a deepened desire to make a difference in the communities we work with, especially for women and children. At the time, a partnership to link various U.S.-funded activities in northern Kenya had just been formed called the Partnership for Resilience and Economic Growth, or PREG. I took the lead to motivate and challenge our development partners and staff to improve the way we do business in northern Kenya so that we have a greater, more meaningful impact on the communities we serve.
As part of the Partnership for Resilience and Economic Growth (PREG), USAID works to improve business models in northern Kenya. Since 2015, the Merille livestock market has added mobile banking and veterinary services, and opened up trade to include vegetables and other products from the surrounding areas.
I am often referred to as “Mama PREG” for my nurturing yet tough-love approach to keeping the partnership focused on continual progress. Small changes I make in the way I think and work have the potential to make a big impact. As W. Clement Stone said, “Big doors swing on small hinges.” This is the mindset I promote and champion in my work, and within the partnership, I challenge all members to leave a legacy in the work we do every day.
Watch Dorine Genga featured in this short video about how USAID is fostering resilience in Northern Kenya via PREG:
The Merille livestock market is an example of PREG’s success. There I was able to directly address the challenges I had heard about on that previous REGAL site visit. In 2015, the Merille livestock market was being rehabilitated. As part of my work, I facilitate joint planning sessions among partners to identify the opportunities for collaboration, and include the needs of the community and the impact on the environment into the design.
With the new design, the community members have access to mobile banking to address security concerns, and commercial banks have even opened offices in the market. The county government started providing veterinary services in the market, making animal disease detection and treatment more accessible so that traders, such as the woman I met, don’t need to travel as far to treat their herds.
Moreover, the market has opened up trade beyond the sale of livestock, with vendors selling hot food, vegetables and other products from surrounding counties. Now, going to the market in this remote area affords community members an opportunity not only to sell their livestock, but also to buy their family’s provisions and seek health care services at the nearby Merille clinic. Partnership has played a major role in making the market functional, bringing transformation to the pastoralist community in Merille and northern Kenya as a whole.
A USAID survey in mid-2015 offered evidence of PREG’s impact. After 2 1/2 years, there was a documented 12 percent reduction in the depth of poverty and a 28 percent increase in women’s dietary diversity in northern Kenya. For me, this information validated USAID’s new way of doing business, demonstrating that collective actions lead to collective impact. At a personal level, the data provided the impetus for me to work to overcome challenges of a maturing partnership and facilitate an environment that promotes continuous learning and improvement.
One of my proudest moments was when I co-designed and facilitated a transformational leadership training workshop for PREG’s county-level leads. One participant stated that his takeaway nugget from the workshop was this: “The essence of a partnership is working together to improve outcomes that improve a community’s well-being.” This workshop demonstrated how we have cultivated transformational leadership among our stakeholders as champions for change—a key component to Kenyans’ ability to transform the arid lands and maximize the sustainable impact of U.S. government investments in Kenya.
“Learning” to Engage the Private Sector in Development
Nov 28, 2018 by Ali HayatComments (0)
This blog post was written by Ali Hayat (Chief of Party, MESP) and Margaret Lada (Monitoring and Evaluation Specialist, MESP).
Private sector engagement is an essential part of addressing development challenges. Key outcomes related to economic development hinge on the ability of the private sector to generate sufficient opportunities, income, and investments. The private sector can also play a role in efficiently providing certain goods (for examples books, food etc.) that contribute to the accomplishment of larger development objectives such as improving the quality of education or enhancing food security. Development interventions are also more likely to be sustainable when there are incentives for the private sector to continue to perform certain necessary roles.
However, the private sector’s ability to play such a constructive role requires an enabling environment and a certain level of development within the private sector. Understanding the profile and maturity of the private sector is therefore an essential first step for efforts focused on engaging the private sector. Such an understanding is also beneficial for figuring out the type of technical assistance that may be provided to the private sector to further enhance its maturity and its ability to positively contribute to development outcomes.
The global economic crisis, regional instability and a skills mismatch between graduates and market demands have slowed Jordan’s economic growth. A rapidly growing population and the influx of refugees are placing additional pressure on the labor market. As a result, unemployment has increased from 12.2 percent in 2012 to 18.1 percent in the first half of 2017. The poverty rate has also increased from 14.4 percent in 2010 to an estimated 20 percent in 2016. In this context, a priority of the donor community lies in the development of the Jordanian private sector and increasing employment. When it comes to the data on the private sector however, up until recently the donor community has had access to a limited number of sources. Even the more systematic and standardized World Bank Enterprise survey, last fielded in 2014, only covers 573 businesses from five out of the twelve governorates of Jordan and therefore does not capture the present and complete realities of the private sector. To address this knowledge gap, the USAID Jordan monitoring and evaluation support platform, Monitoring and Evaluation Support Project (MESP), conducted a survey of the private sector during the winter of 2017.
Private Sector Survey
Starting from a universe of over 80,000 registered Jordanian businesses, the USAID Monitoring and Evaluation Support Project used quota-based non-equal probability sampling that allows for representative data at the company size, sector (based on International Standard Industrial Classification) and governorate level. USAID Monitoring and Evaluation Support Project conducted a survey of 1,864 registered businesses throughout Jordan towards the end of 2017.
The survey is representative of the Jordanian formal private sector at the governorate, business-size, and business-sector levels. The survey questionnaire built on the World Bank Enterprise Survey and captures the perceptions of the private sector on key issues surrounding growth and competitiveness in Jordan, including company performance, employment, access to finance, the business enabling environment, connectedness, and attitudes toward women and youth.
PRIVATE SECTOR SURVEY – KEY FINDINGS
Firms are focused on local markets, with very little national or export activity
92 percent of firms access markets within their own governorate. 37 percent of firms access markets across Jordan. 9 percent of firms export to the Middle East and North Africa (MENA) region. Less than 5 percent of firms export outside of the MENA Region.
Most firms are struggling, but small firms struggle more
62 percent of firms are in poor financial health. 67 percent of micro firms and 57 percent of small firms are in poor financial health compared to 35 percent of large firms.
Access to finance is limited
88 percent of firms state that access to finance is important to their growth and success. However, only 23 percent have applied for a loan.
Employment is erratic, with no clear strategy or pattern
47 percent of firms report erratic hiring patterns.
Women face challenges in engaging in the private sector
Only 16 percent of firms are owned by women. 49 percent of firms report that they do not hire women as employees.
Private sector lacks connection
Only 12 percent of firms work with or are a part of a trade association or other organized business interest groups. Only 35 percent of businesses collaborate with other businesses.
Private sector is underserved
70 percent of firms reported that business support services were not available to them.
Sharing Learning and Fostering Collaboration: Engaging the Relevant Stakeholders
On October 24, 2018, the USAID Monitoring and Evaluation Support Project hosted an event to release the survey data. The event featured an overview of the survey methodology and key findings, followed by an in-depth panel discussion with local business leaders to discuss the real-world implications of the survey findings. The discussions focused on major areas of concern for Jordan’s economic growth and development, including access to finance, employment, firm financial health, market access and gender. Audience members were also encouraged to visit the publicly accessible Jordan Development Knowledge Management Portal and download the dataset, questionnaire, and other materials to use in their own work and program development.
The event brought together over 75 stakeholders from USAID, the Government of Jordan, the donor community, implementing partners, and the private sector to discuss major findings from the survey data. The event was a first step in fostering collaboration between development actors to create data-driven, synergized strategies to engage the private sector and tackle Jordan’s most pressing economic growth and development priorities.
MESP is currently implementing a post-event engagement strategy to ensure continued collaboration, learning and utilization of the private sector survey findings. This includes providing tailored follow-up presentations to USAID implementing partners and other governmental and non-governmental stakeholders, hosting workshops on data analysis and visualizations so that stakeholders have the ability to use and analyze the PSS dataset themselves, and conducting individual consultations with stakeholders on how to integrate this data into program designs and approaches. All the materials and data related to the private sector survey have also been made available on KAMP, the Jordan Development Knowledge Management Portal.
Learning agendas help agencies plan research and evaluation efforts and build a culture of evidence. They “allow agencies to think really critically about the programs and processes and the policies that they’re responsible for,” said Diana Epstein, evidence team lead at the Office of Management and Budget, at a recent Urban Institute event.
During that conversation, federal leaders discussed themes and considerations for developing learning agendas and encouraging the use of evidence within agencies. Here are four key take-aways.
1. “There’s no one-size-fits-all approach”
Several agencies have already developed federal learning agendas and taken different approaches.
“There’s no one-size-fits-all approach,” said Epstein. “Learning agendas should be iterative, flexible, transparent, and most importantly, tailored to meet an individual agency’s needs.”
Differences in agency size, structure, and capacity mean approaches to learning agenda development and content will vary. While some agencies have public, agency-wide learning agendas, others are internal, focus on subagencies, or aren’t called “learning agendas.”
Melissa Patsalides, acting deputy assistant administrator for the Bureau for Policy, Planning, and Learning at the United States Agency for International Development (USAID), said, “Because of our decentralized nature and our spread across sectors, we’ve had a number of learning agendas at USAID over the years” specific to particular programs or offices. Now, USAID is developing a more agency-wide learning agenda focused on a particular policy priority, the Journey to Self-Reliance, or how the agency supports its partners to eventually solve their own development challenges.
2. Learning agendas are integrated into agency functions
As learning agendas fit agency contexts, they should also fit within current agency activities and functions.
Terell Lasane, lead program evaluator at SBA, said the agency “took advantage of that synergy with the performance management functions of our office when we were serving an evaluation function.” When asking program offices about performance management, they also asked about the knowledge gaps that informed the research questions in the learning agenda.
3. Stakeholder engagement is crucial
Learning agendas rely on input from many internal and external stakeholders to ensure the importance and relevance of the questions and research.
Lasane described SBA’s “learning agenda road shows,” where evaluation staff asked program staff what they would change about their program and what questions they would have to answer to make that improvement. Their responses helped inform the Enterprise Learning Agenda, empowering staff to help define the agency’s priorities and making them feel heard, rather than just viewing it as a directive.
One approach ACF has taken is to convene experts in research, academia, and philanthropy to ask “where should [the agency] be headed, what’s going on [in the field] that we want to be sure not to duplicate, what can we complement, [and] are there gaps out there that the federal government is uniquely suited to fill?”
Agencies have leveraged internal and external input in many ways. Recent research provides a playbook for federal agencies, offering common themes and methods for stakeholder engagement, drawn from more than a dozen interviews with federal staff involved in learning agenda development.
4. Learning agendas increase evaluation capacity
Learning agendas provide a long-term strategy for thinking about the role of evidence within an agency and can spur improvements in evaluation capacity. Partnering with program offices, agency leadership, external researchers, and other stakeholders can build buy-in and prioritize research as an important agency function.
Learning agendas also bring people into the evidence space who otherwise may not be as engaged. Patsalides described the importance of including short-term questions that can be answered quickly to demonstrate the agenda’s usefulness.
“The process of developing the learning agenda can be as important as the document itself,” said Maia Jachimowicz, vice president for evidence-based policy implementation at Results for America. “This is not a compliance exercise; it should not be a compliance exercise. It’s really meant to bring meaning to all of our work and help build a culture of evidence.”
Try this Tuesday: Take Five Minutes to Pause & Reflect
Harvard Business School researchers found that “… purposeful reflection on one’s accumulated experience leads to greater learning than the accumulation of additional experience.” This means that to learn, we can’t just consume information and then turn around and implement it. We have to stop and ask ourselves how our efforts are progressing, why and what we should do differently to learn and be more effective.
The video embedded below references a study conducted on employees in a 30-day training program. The study found that employees who took 15 minutes each day to think and journal about what they had learned during that day’s training session performed 23% better on an evaluation than the control group, which did not spend time reflecting.
We often feel we are too busy to pause and reflect on our work, but not reflecting and making necessary changes can affect our performance and ultimately prove more costly. Because reflection translates learning into better decision-making, it can help organizations operate more effectively, leading to better development results.
So, before you power down for the day, take five minutes to reflect on one or more of these questions:
What went well today, and why?
What did not go well? What should I do differently next time?
What did I learn about my operating environment or technical area? Who should I share this information with?
Or, plan to take fifteen minutes during your next team meeting to discuss one or more of these questions with colleagues:
What have we learned from examining our project/activity data? What can/should we do to adjust our work accordingly?
What has changed in our operating environment or technical area? What can/should we do to adjust our work accordingly?
How did this pause and reflect exercise help you? We want to hear about it! Let us know in the comments how this bite-sized activity helped you and/or your team collaborate, learn and adapt.
Three Solutions for Getting Teams to Actually Use their Data
Nov 5, 2018 by Kat Haugh, Amy Leo, Kristin Lindell, Monalisa SalibComments (0)
There is a growing body of evidence suggesting that people struggle to actually use data and evidence to inform their decisions. While there are a number of reasons for this, one of the main reasons is that teams and organizations often fail to internalize the data and evidence they have. If people don’t interpret or reflect upon their data, they are much less likely to use it to inform their decisions. Through our experience, we know that facilitating fun and engaging group reflection is one way to ena ble teams to interpret their data, build team cohesion, and take collective action.
Below you’ll find summaries of three interactive approaches to data internalization that you can apply in your work. The three approaches are relevant to both qualitative and quantitative data and have been used by USAID LEARN to internalize and apply what we’ve learned from our data or help others we work with do the same.
Headlines: Participants internalize data by developing the front page of a newspaper that highlights their key takeaways from the data.
Quotables: Participants internalize qualitative data by matching quotes to the stakeholder group to gain a deeper understanding of various perspectives.
Madlibs: Participants internalize data by selecting the numbers they believe would accurately complete the sentence.
You can find more detailed descriptions and facilitation plans for these approaches here.
We (Monalisa and Kristin) presented these approaches in a Humentum Conference session called "Let's Have Some Fun: Solutions for Actually Getting Teams to Use their Data" and recently heard from a session participant who tried the Headline approach.
“The [headline] session was part of a wider block on capturing and leveraging project learning during close-out. I prepared a data placemat using selected information from an external end-of-project assessment. I divided the participants into three groups. All used the same placemat, but were thinking about different audiences when approaching the data interpretation/headlines exercise... After each group drafted their headlines, we posted them by category and had a plenary discussion of differences and similarities among the headlines and how reflecting on the target audience for the headline impacted the headline writing. The exercise went quite well and we had a very good plenary debrief, despite this being the first post-lunch session on Day 6 of a 6-day training!”
There are any number of approaches that can be used, but these are some of the easiest to understand if you're new to facilitation or helping others internalize data. Let us know if you try any of them, or if they inspire you to design one of your own!
Mapping the Development Journey: Using large-scale surveys to understand context and address learning needs
Oct 23, 2018 by Ali Hayat, Dan KillianComments (0)
This blog post was written by Ali Hayat (Chief of Party, MESP) and Dan Killian (Technical Manager, MSI).
Decision-makers in international development are increasingly inundated with information. Effectively navigating and utilizing this information is complicated by operational and technical challenges. The methodological differences between the available data streams make it difficult to meaningfully aggregate or disaggregate findings. The available information may also present conflicting accounts of the context, without sufficient explanations for understanding “why” this may be or to “what extent” certain findings may be generalizable, and “how” the different findings may fit together. Thus paradoxically, information excess results in a fragmented understanding of the overall context in which decision-makers operate.
Within large USAID missions, such challenges also make it difficult to holistically assess Mission performance and contributions while complicating any understanding of the relationship between donor interventions and a country’s overall development journey. All of this has a significant bearing on the ability of the decision-makers to make informed choices. As the largest donor in country, USAID/Jordan manages over 40 implementing partners and, with the five-year Memorandum of Understanding signed between the governments of the U.S. and Jordan in February 2018, a minimum annual budget of over $1.25 billion through FY 2022.
To enhance a better understanding of the overall context, USAID/Jordan requested its Monitoring, Evaluation and Support Project (MESP) to design a large-scale survey in order to measure mission indicators in the Jordanian population and among self-identified beneficiaries, provide implementing partners with survey data that could potentially be adapted for activity evaluation, and explore a learning agenda to identify and better understand the determinants of indicator performance.
Survey Development and Implementation
The MESP team started with a formal data and document review process, including documents and material on the Jordan Development Knowledge Management Portal (KAMP). This process was followed by semi-structured learning agenda discussions with four USAID technical teams, including Democracy and Governance (DRG), Economic Development and Energy (EDE), Education and Youth (EDY) and Water Resources and Environment (WRE). The purpose of these discussions was to understand the learning needs of technical teams that may be addressed through a large-scale survey. In addition to reviewing existing survey instruments, the MESP team also met with several implementing partners, and the larger US Embassy community.
The final version of the survey has modules on household information, general conditions and public services, education, citizen participation, women and society, employment, entrepreneurship and respondent background. Specifically, the survey covers six mission performance indicators from all of USAID/Jordan’s Development Objectives (DOs), while also measuring a selected number of activity performance indicators. Given the geographic spread of USAID in Jordan, a nationally representative sampling plan was developed with a survey sample size of 12,000. This will allow the survey to provide national and sub-national level context and trends, as well as the ability to disaggregate data based on key demographic variables of interest such as age, sex, income, nationality, and level of education.
Learning Agenda Questions
What are the factors that help determine unemployment or entrepreneurship? What are the greatest impediments to Jordanians starting new businesses? (EDE TEAM)
What is the level of citizen awareness and knowledge of Jordan’s decentralization agenda? At what level of governance does subsidiarity [decentralization] most effectively reside—local councils, municipal councils, or governorate councils? What is the locus of control for all three levels of government? (DRG TEAM)
What is the relationship between citizen engagement and citizen participation? Does stronger engagement and participation lead to improved government effectiveness and legitimacy? (DRG TEAM)
Are migrant population flows disrupting local governance institutions and economic livelihoods? How are local institutions, such as mosques, local meeting groups, and CSOs, adapting and responding to the pressures of migrant population flows? (DRG & EDE Team)
The survey results provide various utilization possibilities:
By providing contextual data at the national and sub-national levels and corresponding to USAID’s DOs, indicators, themes and learning needs, the survey data can inform USAID’s internal portfolio review conversations.
The results from this survey will help answer critical questions of interest to the USAID technical teams to help improve ongoing activities as well as to more effectively design future activities.
The survey can provide a national and sub-national level baseline of key indicators, which can be tracked over time, assuming the survey is repeated. In the case of Jordan, where a new CDCS may be developed soon, this survey could also provide a good baseline of public opinion.
The survey findings can be used by both USAID and implementing partners as baselines for specific activities, as well as to estimate realistic targets by tracking change over time.
The donor exposure questions allow for identifying the type and extent of exposure. Responses from these self-identified beneficiaries provide an opportunity to test theories of change, as well as to set up realistic targets.
Findings from this survey can contribute to the current conversations related to self-reliance by providing tailored country-specific data on general population perceptions and opinions.
Exploration of survey data may generate interesting questions that may be investigated more thoroughly by individual assessments or activity-level data collection.
The findings from the survey will also be useful for a much wider group of stakeholders including USAID, USAID IPs, donors, Government of Jordan, private sector entities, and non-government organizations working in the development realm.
To learn more, find a resource on this initiative and a survey instrument template here.
Failing Forward: Learning from neglected tropical disease programs
Oct 11, 2018 by Katie ZoerhoffComments (0)
Katie Zoerhoff is a Senior Monitoring, Evaluation and Learning Specialist for neglected tropical diseases (NTDs) at RTI International. Ms. Zoerhoff works on ENVISION, USAID’s global flagship project aiming to control and eliminate NTDs.
A young boy has his finger pricked during a transmission assessment survey in Indonesia. Photo credit: RTI International
In global health programs, the stakes are high. Failure is not an option when the end goal is saving and improving hundreds of millions of lives around the world.
But it’s inevitable that, along the winding and complicated road of disease elimination, setbacks will happen. For the past nine years, I’ve been working on neglected tropical diseases (NTDs) at RTI International, where I currently oversee our approach to facilitating collaborating, learning and adapting (CLA) for NTDs on ENVISION, USAID’s global flagship activity for the control and elimination of NTDs. Throughout this time, I’ve learned that a rapid, deliberate, and flexible response to failure is key to our success controlling and eliminating NTDs.
Take lymphatic filariasis (LF), a debilitating NTD that affects more than 120 million people worldwide, as an example. LF is targeted for global elimination as a public health problem; to reach this goal, mass treatment campaigns must be carried out in each endemic district for a number of consecutive years with high coverage. Once this step is complete, a series of surveys called Transmission Assessment Surveys (TAS) are conducted to ensure disease prevalence has fallen below acceptable levels.
Across the 19 countries supported by ENVISION, around 95% of the districts pass these surveys – but sometimes they fail. While we think that’s a pretty great record with more than 625 districts passing a TAS, disease elimination requires a perfect score, an A+. When they pass, we know that the population in those districts no longer needs treatment, a critical milestone on the way to elimination. When districts fail, it’s a signal that something needs to change. So, how do we respond and adapt to these failures?
Here’s what we’ve learned:
Plan for failure. In investigating TAS failures, we started to find ourselves asking some of the same questions – Was the sampling correct? Were the teams well-trained? Was there high enough coverage? As a result, ENVISION worked with the World Health Organization (WHO) to develop a set of checklists to help national NTD programs prepare for and supervise TAS, as well as investigate and respond to TAS failure. By anticipating these issues, teams are able to respond and adapt more rapidly to failure.
Be flexible. To support investigation and the implementation of an adapted approach, we create contingency budgets that can be flexibly allocated to countries that experience failure. Setting aside these resources allows us to respond quickly to needs and adapt our approach, which ultimately saves resources in the long run.
Get deeper in the data. Data collected through routine monitoring and evaluation may be sufficient when activities are going as planned, but additional data may be required to investigate how and why a failure occurred. Start with what you have, but don’t be afraid to do additional (often qualitative) data collection. For example, in areas that fail TAS, ENVISION often conducts interviews with community members, local and national health officials, and other key stakeholders to find out what worked and what didn’t work in the previous year’s treatment campaign.
Fail fast, but don’t rush the L and A. Electronic data collection and analysis enables the quick identification of failures and enhanced visualization. While we aim to learn about issues early and respond quickly, it’s important not to rush the process. ENVISION strives to be intentional about learning from a TAS failure, taking the time needed to do additional data collection, reflect on these new data, and determine the best corrective action to take. At times, we have delayed interventions to ensure we were effectively using our learning.
Communicate openly. Too often in global development, successes are shared but failures are hidden. That’s not our approach. All relevant stakeholders should be aware that a failure occurred, contribute to and benefit from learning, and agree with new programmatic adaptations. For ENVISION, that means we facilitate communication among national and local health officials, community members engaged in mass treatment campaigns, project staff in headquarters and the field, WHO, and USAID.
Of course, we aim to pass in the first place. But, if failure happens, we know we can’t continue business as usual. Using a CLA approach helps us respond quickly and effectively, making efforts to beat NTDs like LF even stronger. The good news is, it’s working, and more than 208 million people in ENVISION-supported countries no longer have to worry about contracting LF. We might occasionally fail, but we are failing forward — seeing incredible progress in the fight against NTDs and learning a lot along the way.
Learning about Learning in the Philippines: Stories from a Cross-sectoral Learning Summit
RTI’s Monitoring, Evaluation, Research, Learning, and Adapting (MERLA) Community of Practice hosted an event titled “From learning to adapting: How do we get to learning, and where do we go from there?” in the Philippines to discuss USAID's Collaborating, Learning, and Adapting (CLA) approach. This summit was an in-country follow up event to a similar Learning Symposium that we hosted in Washington, DC, in May 2018. We wanted to share experiences from the field related to learning and adapting and hold discussions about how we can move beyond traditional M&E practices to a more dynamic learning and adapting model. While USAID missions and partners have long been collaborating, learning, and adapting in various forms, USAID/Philippines recently made its approach to CLA much more intentional by launching the CLA for Improved Health (CLAimHealth) project. Given USAID/Philippines’ long history of cross-sectoral development programming that has adapted through the years with significant contextual changes, we saw an opportunity for rich discussion as the Philippines begins its journey towards learning and adapting. Speakers in the Philippines Learning Summit included:
The event brought together an audience of nearly 140 individuals who contributed to a rich and engaging discussion on CLA, representing a wide spectrum of international development sectors.
Here are a few of our top takeaways from the event:
Collaborate, learn, and adapt - like a camel
During the keynote, Josie challenged participants to be more like a camel. After a pause for laughter and bewildered looks, we heard all the ways that camels are great role models for thinking about CLA. First, camels always travel in a pack. They know they are better off when they work together, or COLLABORATE. Second, the camel depends heavily on its eyes. Camels have three eyelids to protect their eyes from the harsh desert sand, because their eyes lead them to where they need to go. They constantly LEARN from their surroundings and make decisions about where to go. And lastly, they are highly ADAPTABLE. They have evolved to hold fat in their humps as a backup source of energy for when they do not have access to food and water, thereby ensuring their survival.
There is no one right way to do CLA
Throughout the day participants shared some of their successful examples of CLA tools and approaches, including monthly data review guides, pause and reflect sessions, and learning agendas. There is not just one but rather many right ways to do CLA because doing CLA the right way is context specific and context driven. The ENVISION activity discussed how local solutions were key to solving local problems, and that CLA processes, such as learning from M&E data with their Data for Action tool, allowed them to find these context-specific solutions. It is important to think about learning and adapting intentionally and deliberately, rather than as a by-product of the work we do. And in order to do CLA intentionally, we need concrete tools and approaches that are tailored to the local context.
It’s time to address the myth that CLA is expensive and only donor-driven
USAID’s STRIDE activity showed us that the private sector is interested and can invest in projects when CLA is used to increase trust. STRIDE began to connect universities and private industry, but the relationships were forced and untrusting. Through intentional learning and review of activity programming, STRIDE adapted and increased the number of times universities and industry met, instead of holding a few big events a year. This collaboration paid off, as the relationships grew stronger with increased contact. Eventually, the private industry took over the investments that USAID was making, and worked directly with universities. STRIDE used CLA processes (such as pause and reflect sessions) to increase trust and collaboration between the private sector and universities. While incorporating CLA into an existing or new program requires initial investments and funding, once the value of CLA is seen through shared results, it is easier for CLA investments to be leveraged from donors and the public and private sectors. We need to continue to gather evidence to show that CLA can be sustained beyond donor funding as it can lead to tangible benefits for stakeholders.
CLA is effective development
When CLA is carried out through a truly collaborative spirit with ownership by host country governments and local stakeholders, it is not just good and inclusive international development, it is also effective international development. As governments and local stakeholders become equal partners in international development efforts, the achievements of international development programs become more sustainable. As we model and teach CLA practices in our work, they can then be carried forward by local governments and stakeholders to achieve better development outcomes.
Through the discussions and Q&A many other points emerged that deserve further exploration, including how to quantify and show return on investment for CLA and the need to develop best practices for identifying and building CLA champions with programs and local governments. Stay tuned for more on this from the RTI MERLA Community of Practice (MERLA@rti.org), #RTILearns.
The Zine: How to Use Your Visual Brain to Improve Reporting & Intentional Learning
Aug 29, 2018 by Kat HaughComments (0)
Kat Haugh is a Monitoring, Evaluation, Research, and Learning Specialist on the USAID LEARN contract.
We all have them: that 50-page report sitting on your desk that you’ve never read. And when you think about reading it, you cringe. You just can’t do it. And there’s a good reason for that, research and experience show that in order for information to be used for learning and adapting, the information itself needs to be easily accessible.
How do we present information in a way that capitalizes on opportunities for information to actually be digested and used (and avoid the dreaded 50-page report)?
One way the USAID LEARN contract makes data more accessible and actionable is through visual storytelling. We use metaphors, doodles, storylines, and designs to communicate about our work and better understand what our data is telling us about our progress and impact.
We’ve recently piloted a new visual storytelling technique called a zine. The term zine is short for magazine and is a self-published visual story with a combination of text and images. Zines are a visual reporting tool used to distill complex information (like information about financial investment or climate change) into short, easy-to-read, creative stories. Everyone can be an author (and also an editor, designer, and publisher) of a zine, and that is part of what makes them so powerful.
We got inspired by zines and so we created one about our work on LEARN. The finished product looks like this:
If you can’t tell from the picture, the Zine is a small (8.5 x 11 inches) booklet that includes minimal text and lots of visuals. It’s written in story-form, with no jargon or flowery words. That’s because the focus is on communicating the information as succinctly and efficiently as possible and presenting it as a story. We do that because our brains not only make sense of visual information more quickly, but we are also far more likely to remember stories.
On our journey to creating a zine about our work, we learned a few key lessons:
Need to know > nice to know. Given the limited space in the zine, we had to be really strict about what was absolutely necessary to communicate and what was not.
Ideal visual to text ratio. To avoid overdoing it with the amount of visuals in the zine and overwhelming our audience, we followed guidance from data visualization experts like Stephanie Evergreen and others about the appropriate visual to text ratio.
Co-creation of recommendations is helpful. In order to make the content of the zine as useful as possible, we worked within our team to co-create the recommendations about changes we’d make in our work in response to the data that was reported in the zine.
Facilitation is needed for internalization. Just sharing the zine is not enough for the information to be adequately digested and used for decision-making. Facilitating the internalization of the information through an exercise (like a madlibs game) is critical for the information to be downloaded by your team.
Like all types of reporting, the zine comes with its own pros and cons. The pros are that this type of reporting focuses on what is most important, is told in story-form, and is highly visual and engaging. The cons are that the zine cannot always capture the details and the visuals can feel overwhelming to some.
There is a lot of opportunity in our field to improve the way we are communicating about our work. We’d love to hear how you are using your visual brain to do it! Please share your stories with us. And, click here to see one of our zines.