The use of Indicators for the Monitoring and Evaluation of KM in International Development

Update:

This project is now successfully closed. The survey report may be found at: http://www.km4dev.org/profiles/blogs/measuring-knowledge-management-survey-report-for-the-km4dev

---

This proposal has been funded. Please find below a link to a survey on indicators for Knowledge Management (KM) and Knowledge Brokering (KB) that will take 5-10 mins to take part in:

https://www.survey.lboro.ac.uk/k4devindicators/

This will feed into a project and workshop which seeks to build an overview of how KM and KB activities are being measured and what indicators are in use.

We would be very grateful for your input in completing the survey and your help in sharing / tweeting (#K4DevIndicators)/ blogging on, and otherwise disseminating this survey within your knowledge networks

Kind regards, Walter Mansfield and Philipp Grunewald (Loughborough University, UK) Yaso Kunaratnam and Louise McGrath (The Institute for Development Studies, UK)

Background information

This survey has been produced for this KM4Dev Innovation Fund project and as part of a larger collaborative project between researchers from Loughborough University and the Institute of Development Studies (IDS), funded by DFID as part of the Mobilising Knowledge for Development programme. The survey will be used to assist in the development of a workshop on Knowledge Management and Knowledge Brokering indicators to be held on the 8th March at IDS, UK. The results and findings of the survey and workshop will be reported back to the KM4Dev forum and the Knowledge Brokers Forum (KBF).

Summary description

Applying Monitoring and Evaluation (M&E) methods to knowledge management (KM) has been described as "an impossible task" with "no systematic approach" (Simon Hearn IKM Emergent presentation). However, without an ability to measure and demonstrate impact, knowledge management is facing a struggle for legitimacy.

Developing effective KM Indicators would allow KM activities to be externally scrutinised, evaluated, and held Accountable (enabling practitioners to demonstrate the impact of KM, providing an evidence base to justify investment). They would also enhance Learning (allowing practitioners to determine which approaches to KM are more effective, enabling improvement within organisations and across the sector).

Whilst recognising the challenges of measuring knowledge, which is to a large extent intangible and unquantifiable, it is contended that proxy indicators can be designed to improve the effectiveness of KM initiatives, to legitimise KM within the sector and to improve the impact of international development on poverty and inequality.

Research question

Support from the KM4Dev innovation fund will assist in the following research investigation: What indicators are being used to measure Knowledge Management in international development? The research aim is to assemble a pool of KM indicators and to assess their appropriateness through a facilitated peer-review process.

Research Methodology and main activities

The research will draw upon existing indicator models both within and outside of the field of international development; engage with development and knowledge management practitioners utilising existing communities of practice; facilitate participatory activities centring on a practitioner workshop in which a pool of indicators will be developed and critiqued. Expected outcomes will include a) a compilation of KM indicators currently in use; b) a facilitated peer-review of the strengths and weaknesses of these indicators c) a report summarising findings and containing suggestions for further piloting d) a proposed sub-forum community of practice within KM4Dev for continued engagement beyond the time frame of this project. The results of these engagements will be shared with the development community in a variety of ways including the dissemination of a report and feedback sessions.

Relevance for the KM4Dev community

IKM Emergent has explored the use of indicators in its working paper series. Hearn et al (2011) argued for indicators which reflect the "nuanced understanding, interpretation and innovation" of knowledge management and sharing activities, and Ferguson et al (2008) called for a re-definition "of the concept of ‘impact’ and the indicators for its measurement". Ferguson et al's paper also posed the following research question "What indicators have been developed to measure the impact of knowledge management for development?". While there has been a growing debate on the subject of measuring KM impact, including whether impact measurements are helpful or even necessary, the question of what indicators are in place remains unanswered.

This research is relevant since it brings together knowledge holders on an issue which is at the core of the impact debate of KM for development. The focus on indicators will enable the project to generate new insights and also lessons for practitioners in the field. Members of the community will benefit from exposure to a broader base of indicators in use by sector actors and also from the reflections that will emerge in the process (from both academic and practitioner perspectives).

Additionally, this work could act as a trigger to create a set of replicable indicators for use across multiple programmes. This could allow for easier comparisons regarding the effectiveness of interventions and the community could gain a greater understanding of the strengths and weaknesses of different types of interventions in different contexts.

Bibliography

Ferguson et al, IKM Working Paper No. 1, Meta-review and scoping study of the management of knowledge for development, March 2008

Hearn et al, IKM Working Paper No. 12, Monitoring & evaluating development as a knowledge industry, 2011

Activities and funding requirements

Requested funds total of $3,800 [amended to $1400 due to alternative funding]

Indicative breakdown of costs:

Transport and accommodation - $600 (to potentially include transport costs for pre-workshop meetings and workshop. Transport and accommodation costs to present research findings.)[KM4Dev Innovation Funding no longer requested]

Development and creation of research outputs - $1400 (Development, design and publishing of findings report.)

Workshop costs - $1800 (of which room and equipment hire / refreshments $1000. Research, communications and facilitation $800.)[KM4Dev Innovation Funding no longer requested]

Background and contacts

We are two PhD researchers at Loughborough University with backgrounds in international development and knowledge management. As part of his wider PhD research Walter Mansfield (research lead) is seeking to develop an effective framework for measuring knowledge management in international development NGOs to target the two main strands of M&E: Accountability and Learning. Philipp Grunewald (research facilitator) is looking at the facilitation of knowledge exchange in international development and blogs at: [1]. Walter is supported by a research stipend jointly from the UK AHRC and Loughborough University. Philipp is in receipt of a studentship from Loughborough University.

This fund would assist us in meeting some of our costs associated with this specific investigation and would also contribute to our wider research aims.

We are both active members of the Loughborough University Knowledge Management Research Group: [2]

Walter Mansfield W.Mansfield@lboro.ac.uk

Philipp Grunewald P.Grunewald@lboro.ac.uk