What to Expect When You're Expecting CLA: Lessons from the CLAIM Learning Network

Jun 26, 2018 by Larissa Gross Comments (2)
COMMUNITY CONTRIBUTION

This blog was written by Larissa Gross, Senior Strategist at the Pollen Group, on behalf of the CLAIM Learning Network.

What difference does collaborating, learning, and adapting (CLA) make to development? This is a question USAID/PPL and its partner LEARN have been trying to answer and over the past 18 months, and recently set out to expand the knowledge base on CLA with the help of five implementing organizations testing the impact of CLA interventions across activities in different parts of the world.

So what? What did we find?

  1. CLA practices and organizational and developmental outcomes must be clearly defined from the outset to facilitate measurement.
  2. When measuring CLA's contribution: pivot logs are useful for identifying adaptive actions, CLA capacity self-assessments can garner partner buy-in, and detailed theories of change can help mitigate confirmation bias.

What to Expect When Practicing CLA
Based on our research, we compiled insights into what CLA may look like for different organizations and what to expect when you invest in CLA. The key findings members shared are:

  • One size does not fit all so get ready for some self-reflection. Depending on the timeframe, resources, team members, activity goals, and other unique factors specific to your effort, CLA and the organizational processes, operations systems, and culture that support it will (necessarily) vary.
  • Investing in CLA can uncover turbulence - and that’s not a bad thing! It is important to recognize that organizations beginning to practice CLA will need to test out which approaches work best for their needs. Along the way, organizations will need to navigate a process of learning and adapting based on imperfect information - this is part of the process of better understanding the context in which you are working.
  • People will see that they are already doing it. CLA may look like existing practices within activities. If it looks familiar, then we are on the right track! Lean into this familiarity when promoting CLA by connecting to what teams are already doing.

How to Measure CLA’s Contribution to Development Outcomes

When the learning network members gathered together to discuss our key learnings, we were asked to provide a main take away on measuring CLA; the headline of our 18 months of research. The common sentiment emerging from this discussion was that measuring CLA’s contribution is difficult, but it is possible! And, it can be done better.

Building on this sentiment, we pulled together key insights on the main challenges of measuring CLA, what methods and tools helped us to measure CLA better and how we would tweak our methods for future research. In particular, the learning network members emphasized:

  • Defining what CLA practices are, how they are interrelated, and which practices will be measured are key to understanding the power of your findings and comparing with others’ findings. Generating a general theory of change and a detailed theory of change can help to minimize the difficult of comparing across research agendas.
  • Methods that analyzed decision-making processes were most successful in connecting CLA to development outcomes but come with limitations including recall and confirmation bias as well as an inherent focus on actions taken rather than those not taken.
  • Flexibility of research design is necessary due to the complexity of the research agenda. Network members recommended, to increase the likelihood of producing robust findings, including multiple partners and using varying tools to increase the adaptability of the research agenda.  

Background on the CLAIM Learning Network

The CLA Initiative for Measurement (CLAIM) learning network launched in the fall of 2016 to answer the following questions: Does an intentional, systematic and resourced approach to collaborating, learning and adapting contribute to development outcomes? If so, how? And under what conditions? How do we know?

The five implementing partners in the network are:

  1. Counterpart International—focused on the Participatory Responsive Governance—Principal Activity (PRG-PA) in Niger to measure the degree to which staff use CLA-generated knowledge and learning in planning activities and executing decisions in their daily work and the degree of empowerment that participants feel they have in those activities.
  2. The Global Knowledge Initiative—pursued replicable approaches for monitoring and evaluating collaboration by testing and refining the Context-Collaboration-Program Effects (CCPE) Analysis on the Learning and Innovation Network for Knowledge and Solutions (LINKS) program in Uganda.
  3. MarketShare Associates—built and tested a set of CLA-focused tactics such as coaching modules on adaptive management and pivot logs through the DFID-funded Arab Women’s Enterprise Fund (AWEF) in Egypt.
  4. Mercy Corps—field tested promising techniques for promoting adaptive management through pilot projects as part of the Analysis Driven Agile Programming Techniques (ADAPT) initiative.
  5. Pollen Group—conducted two comparative, longitudinal case studies of projects that have made significant investments in CLA in Bangladesh and Zambia.

COMMENTS (2)

Evidence Evolves! From CDIE to Evidence in Action

Jun 20, 2018 by Diane Russell Comments (0)
COMMUNITY CONTRIBUTION

A personal history by Diane Russell, Senior Social Scientist, E3 Forestry and Biodiversity Office.

Twenty-five years ago I started a position as Research Manager within the Research and References Services (R&Rs) project, a contract that formed part of USAID’s Center for Development Information and Evaluation (CDIE). This position involved managing teams of over 20 research analysts covering all the Bureaus and sectors. The R&Rs team provided missions with literature reviews and research analyses through cables, and copies of documents through diplomatic pouch. The internet and email were new. One day John Erickson, then Director of CDIE, announced that he had successfully sent an email with an attachment. We all cheered. The library was part of CDIE as was a team of evaluation specialists and of course what is now called the DEC, which housed documents in hardcopy and microfilm. With CDIE, USAID was the recognized global leader in knowledge management for development. I am particularly proud of a series of papers on sustainable development my team produced to help USAID better define and operationalize that concept.

Fast forward to today when we have a wealth of technology to help us search for evidence to support our project designs, bolster our evaluations and build institutional learning. Despite these advances, I would argue that better technology has not created a culture of identifying, using and generating sound evidence. In fact technology can beguile us into thinking that we have done our due diligence. We are overloaded with information and can’t easily discern what is most relevant and of highest quality. We can google and find a few articles that bolster our assumptions about what works and doesn’t work. We rely on indicators available in our databases rather than using them as a springboard to ask questions about the complex causes of trends that affect our work, some of which may stretch back into history while others are so new that only messaging on mobile phones can capture them.

I’ve been working on the evidence base for our biodiversity conservation programming since joining the Forestry and Biodiversity Office in 2005. We’ve produced a research agenda for our sector and a number of great evidence products that cover both goals of the Biodiversity Policy: supporting effective conservation in priority places and integrating conservation as an essential component of development. Uptake of the evidence in some of these products has been high due to their integration into facilitated Learning Networks. Other products are recognized to provide quality evidence but uptake has been slower because programming decisions fall outside of the biodiversity sector. For example we demonstrate that working on wild capture fisheries is critical for food security in many areas. Despite this evidence, investment remains low.

After years of developing these evidence products we came to the conclusion that it is critical to support USAID staff to identify, use and generate evidence in their own contexts and for their own needs. Ideally we could go back to the CDIE model and provide “research analysis on demand,” but setting up such a service is daunting. (Hint: it would be a great investment for the Agency to make!)

What we have done, through our Measuring Impact activity, is produce Evidence in Action, a resource for USAID staff to support an evidence-based approach to biodiversity conservation programming. We believe this resource will be valuable for all sectors; indeed taking an evidence-based approach to biodiversity conservation was inspired by efforts in medicine, public health, and education. It has achieved new urgency given the continued decline in biodiversity that may have pushed the earth beyond “safe” limits for biodiversity loss. This urgency means we must deploy conservation dollars effectively, building from and building up the evidence base.

You may be asking “what do you mean by evidence anyway”? How is it different from data, evaluation findings, knowledge management, learning, or action research? Evidence in Action can answer these questions and more. Evidence can be derived from any of these inputs to craft a body of facts to support or dispute a development hypothesis or theory of change.

But most critically Evidence in Action describes how we can adopt an evidence-based mindset that infuses our work, beefing up our due diligence as stewards of U.S. taxpayer funds and motivating us to move out of our cubes and into the stimulating community of evidence.

What is the Role of Evidence and Data in Organizational Learning Efforts?

Jun 19, 2018 by Piers Bocock, Stacey Young Comments (0)
COMMUNITY CONTRIBUTION

The focus of Leaders in Learning episode 3 is on one of the most challenging questions that we as leaders in organizational learning face on a regular basis, namely, what is the role of evidence and data in organizational learning efforts? This is a complex topic, and one that a single podcast episode can’t possibly do appropriate justice.  But hopefully we bring in some interesting perspectives during this podcast to add to the large amount of content we have on the subject on USAID Learning Lab.  

The contributing thought leaders for this episode are:

  • Alison Evans, Chief Commissioner for the UK’s Independent Commission for Aid Impact (ICAI)
  • Kerry Albright, Chief of Research Facilitation and Knowledge Management at UNICEF
  • Duncan Green, Senior Strategic Advisor for OXFAM

The three themes that emerged in response to the question about the role of evidence and data in organizational learning that we discuss in this episode are:

  1. While everyone seems to agree that data and evidence play a vital role in organizational learning, and making the case for investing in it, approaches and perspectives vary.
  2. That though we may approach it from different angles, all the guests seemed to agree on the importance of an organizational culture that values evidence and learning.  And
  3. That while data and evidence are of course important for accountability, they are absolutely essential for continuous learning and ultimately, impact.

We hope you find the discussion interesting and encourage you to listen to episode 4, in which we discuss the connection between organizational culture and organizational learning.

You can stream new episodes here on USAID Learning Lab or search for “USAID Learning Lab” wherever you listen to podcasts.

Introducing Leaders in Learning, a New Podcast Series from USAID Learning Lab

Jun 13, 2018 by Piers Bocock, Stacey Young Comments (0)
COMMUNITY CONTRIBUTION

Evidence and experience show that leadership plays a key role in creating and sustaining organizational learning and knowledge management initiatives. So what does it take to be an effective leader in learning?  

USAID Learning Lab is launching a new podcast series this summer to explore this question. Leaders in Learning is a 7-part podcast series hosted by Stacey Young, Senior Learning Advisor and Collaborating, Learning, and Adapting Team Lead in USAID’s Bureau for Policy, Planning, and Learning; and Piers Bocock, Chief of Party for the USAID LEARN contract. The series draws on interviews that Stacey and Piers conducted with a variety of thought leaders in organizational learning and knowledge management in the international development sector, including:

  • Kerry Albright, UNICEF
  • Rob Cartridge, Practical Action
  • Chris Collison, Knowledgeable
  • Alison Evans, Independent Commission for Aid Impact
  • Duncan Green, OXFAM
  • Gwen Hines, DFID
  • Clive Martlew, DFID
  • Karen Mokate, Inter-American Development Bank
  • Tony Pryor, USAID’s Bureau for Policy, Planning and Learning
  • Thom Sinclair, The World Bank

Episodes 1 and 2 are now available to stream here on USAID Learning Lab or wherever you listen to podcasts. Listen to the first episode for an overview of the series including its purpose, theory of change, and to meet the lineup of the experts whose interviews provide the food for thought in each episode. Click here to view the Episode 1 transcript.

In Episode 2, Piers and Stacey reflect on their conversations with Duncan Green, Gwen Hines, and Karen Mokate (pictured with Piers and Stacey, right) on the topic of why learning matters in international development. They frame learning as a tool for addressing complexity, improving the way teams and organizations work, and, most importantly, achieving better development results. Click here to view the Episode 2 transcript.

Join Stacey and Piers for the rest of the series as they grapple with these learning questions:

You can stream new episodes here on USAID Learning Lab or search for “USAID Learning Lab” wherever you listen to podcasts.

We hope you enjoy it!

Stopping to Think: Why it Pays to Pause and Reflect (Inside Out Episode 5)

May 30, 2018 by Amy Leo Comments (0)
COMMUNITY CONTRIBUTION

Episode 5 of From the Inside Out: Achieving Better Development Outcomes through Collaborating, Learning and Adapting is now available! Stream it above or subscribe to the USAID Learning Lab podcast wherever you listen to podcasts to be notified when new episodes are available.

In the final episode in the Inside Out series, we discuss a topic that we’ve touched on throughout the series: Pause and Reflect. Taking the time to Pause and Reflect is an essential CLA practice that leads to evidence-based decision making and adaptation.

First, we’ll explore evidence that group reflection leads to learning and decision-making. Next, we will discuss two opportunities for USAID staff and partners to use Program Cycle processes to pause and reflect. In our third segment, we discuss some of the ways that our team has built Pause and Reflect moments into the way we work.

Resources mentioned in this episode:

While this series is wrapping up, we’ve got another one in the works! Listen until the end of this episode for a preview of our upcoming series, Leaders in Learning, which will be released starting in mid-June 2018.

The information in this series comes from our effort to build the evidence base for collaborating, learning and adapting. If you’re interested in learning more about this area of work, visit USAIDLearningLab.org/eb4cla.

Staying in the Loop: How to Learn from your Data (Inside Out Episode 4)

May 22, 2018 by Amy Leo Comments (0)
COMMUNITY CONTRIBUTION

Episode 4 of From the Inside Out: Achieving Better Development Outcomes through Collaborating, Learning and Adapting was released today! Stream it above or subscribe to the USAID Learning Lab podcast wherever you listen to podcasts to be notified when new episodes are available.

Evidence-based decision-making is a powerful tool for transforming USAID from the Inside Out. In this episode, we’ll talk about where we see this happening in other industries, and why it can be challenging at USAID. We will also discuss two examples of innovative learning initiatives at USAID missions.

Resources mentioned in this episode:

We’re releasing a new episode each Tuesday in May, so subscribe to the USAID Learning Lab podcast wherever you listen to podcasts (iTunesStitcherPocketCasts) to find out when new episodes are available. You can also find them posted as blogs here on USAID Learning Lab.

The information in this series comes from our effort to build the evidence base for collaborating, learning and adapting. If you’re interested in learning more about this area of work, visit USAIDLearningLab.org/eb4cla.

Meeting Strategically: How to Collaborate, But Not Too Much (Inside Out Episode 3)

May 15, 2018 by Amy Leo Comments (0)
COMMUNITY CONTRIBUTION

Episode 3 of From the Inside Out: Achieving Better Development Outcomes through Collaborating, Learning and Adapting was released today! Stream it above or subscribe to the USAID Learning Lab podcast wherever you listen to podcasts to be notified when new episodes are available.

Research says that 20-35% of value-add collaboration comes from only 3-5% of employees. As a result, those high collaborators score low on engagement and career satisfaction. In addition, the lion’s share of collaborative work tends to fall on women, and people of color are often over-burdened by requests to collaborate (citation).

In this episode, we discuss evidence on the negative effects of over-collaboration and reflect on our teams' collaboration. Next, we talk about six strategies for strategic collaboration with examples from USAID programs.

Resources mentioned in this episode:

Bonus! We've created a coloring page to go along with the episode. Listen with your team and tweet photos of your colored-in pages to @USAIDlearning.

We’re releasing a new episode each Tuesday in May, so subscribe to the USAID Learning Lab podcast wherever you listen to podcasts (iTunesStitcherPocketCasts) to find out when new episodes are available. You can also find them posted as blogs here on USAID Learning Lab.

The information in this series comes from our effort to build the evidence base for collaborating, learning and adapting. If you’re interested in learning more about this area of work, visit USAIDLearningLab.org/eb4cla.

Four Tips for a Winning CLA Case Competition Submission

May 14, 2018 by Amy Leo Comments (0)
COMMUNITY CONTRIBUTION

These four tips address the most common issues we see in Collaborating, Learning and Adapting (CLA) Case Competition submissions. So, in the final weeks before the deadline (Thursday, May 31!), take advantage of these tips to help your case stand out.

1. Call It What It Is

Collaborating, learning, and adapting is just one of the many terms used to describe adaptive approaches to development, and we do not penalize submissions that describe CLA approaches without calling them CLA. However, there are some important differences between CLA and market systems approaches, doing development differently, and thinking and working politically, just to name a few. If you’re describing a specific, named system or approach that relates to CLA, be transparent about that and explicitly point out the interconnections between CLA and that approach. Don’t simply equate it with CLA without explanation.

Example: Catholic Relief Services' 2017 Winning Case Making Connections, Measuring Results: CLA in a Food Security Program in Zambia clearly describes the connections between CLA and an Evaluative Thinking (ET) approach they took in their activity, outlining how the skills and mindset encouraged by ET also strengthened CLA, and particularly the ability to adapt.

2. Focus on the CLA Aspects of your Case

Descriptions of technical work should only set the stage for your case, not take it over. In your responses, be sure to use the majority of the space in your submission form to describe theCLA approach itself. Don’t get into the weeds describing aspects of your project that don’t relate to your CLA approach.

Pro Tip: If you’re writing about the implementation of a new tool, be sure to focus on how the tool is used for collaborating, learning and/or adapting not the details of the implementation itself. Winning cases describe comprehensive, ongoing CLA.

Example: See The Manoff Group’s response to Question 4 in their 2017 Winning Case Stop, Reflect, Improve: Using CLA to Engage Men to Improve Women and Children's Health. They describe when and how they knew they needed to adapt their project, as well as all of the pivot points and decisions that contributed to its new direction. Any details about their project only serve to provide context for their CLA approach.

3. Connect the Dots

Question 2 of the 2018 Case Study Submission Form asks the submitter to describe the organizational or development challenge that prompted them to collaborate, learn and adapt. Strong cases clearly connect this challenge with their CLA approach, and then, in Questions 5 and 6, connect their CLA approach with outcomes. Be sure that this story arc comes through in your case.

Pro Tip: Winning cases describe comprehensive, ongoing CLA.  If you’re writing about a conference or other one-off event, be sure to strengthen your case by explaining how the event fostered CLA in a sustained and ongoing way.

Example: CARE’s 2017 Winning Case Practice What You Preach: A Tool for Staff Transformation clearly communicates the challenge at hand, what the team did to address it, and the results.

4. Provide Enough Detail

Winning cases will be featured as examples for others to follow, so step back and determine whether your submission provides enough detail for a reader to replicate your approach. Too much detail, however, may make your case less relatable for others. The guiding questions under question 4 are meant to help you determine what level of detail is needed--if you address each of them in the space provided, you’re likely on target.

Example: USAID/Jordan’s 2017 Winning Case No One Can Know Everything: Collaborating for Better Evaluation Recommendations describes the decision-making process surrounding their CLA approach and states who was involved in each step. The submitter also utilized all of the allotted space to include this level of detail. Go ahead and fill the page!

Staying Curious, Together: How to Create a Learning Culture (Inside Out Episode 2)

May 8, 2018 by Amy Leo Comments (2)
COMMUNITY CONTRIBUTION

Episode 2 of From the Inside Out: Achieving Better Development Outcomes through Collaborating, Learning and Adapting was released today! Stream it below or subscribe to the USAID Learning Lab podcast wherever you listen to podcasts to be notified when new episodes are available.

In the first episode of the series, we talked about the characteristics to look for when building an adaptable, learning-focused team.

In this episode, we’ll discuss the key ingredients in a learning culture. First, we define what we mean by a learning culture. Next, we look at research that examines the relationship between CLA and measures of staff satisfaction and empowerment at USAID. And in our third segment, we talk about a successful staff empowerment initiative at a USAID mission.

Episode 2 Coloring Page

Bonus! We've created a coloring page to go along with the episode. Listen with your team and tweet photos of your colored-in pages to @USAIDlearning.

Resources mentioned in this episode:

We’re releasing a new episode each Tuesday in May, so subscribe to the USAID Learning Lab podcast wherever you listen to podcasts (iTunesStitcherPocketCasts) to find out when new episodes are available. You can also find them posted as blogs here on USAID Learning Lab.

The information in this series comes from our effort to build the evidence base for collaborating, learning and adapting. If you’re interested in learning more about this area of work, visit USAIDLearningLab.org/eb4cla.

Going Beyond Technical Skills: How to build an adaptive team (Inside Out Episode 1)

May 1, 2018 by Amy Leo Comments (0)
COMMUNITY CONTRIBUTION

Episode 1 of From the Inside Out: Achieving Better Development Outcomes through Collaborating, Learning and Adapting was released today! Stream it above or subscribe to the USAID Learning Lab podcast wherever you listen to podcasts (iTunesStitcher) to be notified when new episodes are available.

Synopsis:

If change starts with people, then what should we consider when building collaborative, learning-focused teams? When development isn’t a linear process and success depends on much more than a set of technical skills, what kinds of skills and qualities should you look for when hiring?

In this episode, you’ll hear from three development practitioners with insight on this question. (And if you’re wondering how to build a learning culture on your existing team, we’ve got you covered. Be sure to tune in to our next episode, released on Tuesday, May 8, which is on this very topic.)

Coloring Page GIF

Bonus! We've created a coloring page to go along with the episode. Listen with your team and tweet photos of your colored-in pages to @USAIDlearning.

Resources mentioned in this episode:

Share your thoughts on the episode by commenting below. And if you like the podcast, please rate it in iTunes. This will help people like you find it!

We’re releasing a new episode each Tuesday in May, so subscribe to be notified when new episodes are available. You can also find them posted as blogs here on USAID Learning Lab.

The information in this series comes from our effort to build the evidence base for collaborating, learning and adapting. If you’re interested in learning more about this area of work, visit USAIDLearningLab.org/eb4cla.

Pages

Subscribe to RSS - amyleo's blog