You are here
In July 2008 the Institutional Knowledge Sharing (IKS) Project commissioned RE4D.net (Ben Hack) to conduct an independent evaluation of the first phase of the project. This study assesses the results of the four pilot activities, illustrates the systemic impact of the project, and presents lessons distilled from the combined experience of KS professionals in six CGIAR centers. The evaluation study used semi-structured, open-ended telephone interviews to gather feedback from 14 KS practitioners directly involved in the first phase of the KS project. This anecdotal feedback was then organized and analyzed so as to shed light on the benefits and challenges of the KS Project. The interview questions were designed to probe for different kinds of benefits associated with the projects’ overall objectives that practitioners might find in their KS work. The study did not include a more social impact assessment with a full analysis of the literature and primary data gathering. The present study did not have the resources for that; it aimed only to solicit and analyze feedback from the CGIAR KS community involved in Phase 1 of the project. The final report was published in May 2009.
- Even if you can document and showcase behavioral changes, how do you assure that the organization takes them seriously into account and acts on the results? - How do you combine effectively economic approaches to impact assessment and qualitative approaches? How do you make both equally important? - Development projects are more and more often based on innovation theories which include theories of change and other methodologies that don't fit with pure data gathering: How do you convince donors to make the effort to read and analyze other types of information beyond pure data?
The ICT-KM Program team decided at a team retreat that it would be beneficial to undertake such a study in order to "walk our talk" and provide feedback that would help adjust activities and strategies for the second phase of the project. So it was definitively done in order to support our internal learning process.
The interactions with the consultant revealed that the consultant was a good choice as he had knowledge about the CGIAR, M&E processes and KS tools and methods. The interviews worked well, interviewees took the time to respond and have a longer in depth conversation. The size of the project (500 000 USD for 2 years) and the number of people directly involved didn't require in my view a bigger type of analysis at that time. The problem emerged when the consultant had to deal with the "unstructured information" he had received from the intreviews and when he had to identifiy the issues and synthesize the feedback. At that point we had a longer conversation with a senior M&E specialist who was the lead coordinator of the first phase of the project. That helped a great deal to design the strategy for exploring the interview results and nail them down to some really useful conclusions, key lessons and recommendations.
We did the right thing at that time and I am happy to have supported the consultant in the difficult moment he had at some point. You could have argued that we needed a more senior consultant but I still think that our choice was concordant with our willingness to support younger talents. If I had to redo a study today I would include some more quantitative data gathering and analysis which would make sense today as the post projects impacts are increasing through the ICT-KM efforts and interventions. I am currently starting with my team (Capacity Strengthening and Knowledge Management at CIAT) an assessment activity on the impact of CIAT's capacity strengthening efforts and we will use an approach that combines qualitative and quantitative approaches.
The CGIAR’s Knowledge Sharing Project was part of its Information and Communication Technologies and Knowledge Management (ICT-KM) Program. Its first phase, from 2004 to 2006, was designed to foment a knowledge sharing culture within the CGIAR. Coordinated by the International Center for Tropical Agriculture (CIAT), the project design originally centered on the development of comprehensive knowledge management (KM) strategies in CGIAR centers and programs, together with complementary studies and capacity building. With support from Bellanet International Secretariat, the project design was reviewed and then shifted in the direction of introducing knowledge sharing (KS) approaches into major center events in order to involve large numbers of managers and staff and foster capacity building. The project had four main objectives: (1) to review experiences with KS, (2) to generate commitment to introducing KS approaches and tools, (3) to support the development and implementation of KS strategies, pilot activities, and support policies, and (4) to facilitate access to KS tools and techniques. During the two-year project period, and among other activities, four CGIAR centers actively tested KS approaches and tools. Pilot initiatives resulted in the centers organizing and conducting their annual staff meetings differently, contributed to the better integration of a team of scientists who share knowledge and information and work towards common goals and helped one center to launch its own project on KS in research. The KS Project aimed to demonstrate how centers could, by creating opportunities for large numbers of staff to connect with one another, draw more fully on the collective knowledge of their staff and, as a result, plan and execute their work more efficiently.
- Plan a KM assessment at the beginning and not just at the end as an after-thought. This will guide your thinking and acting throughout the project and facilitate the assessment itself. - Don't overdo it and choose the right size and method for the assessment in concordance to your budget and the number of people involved. Small can be smart. - Use many different media and your networks to share the results and start a discussion around those - In case of a small and qualitative assessment don't forget to involve your impact assessment unit nonetheless in order to get conversations going between ex-ante and ex-post, rigorous and other flexible approaches to assessment.
This case was submitted as part of the KM Impact Challenge in 2011. The challenge was sponsored by USAID's Knowledge-Driven Microenterprise Development project, as a key part of the project’s Assessing & Learning component, which sought to improve the understanding of how investing in learning can increase and extend the overall impact of USAID's development efforts.