Talk:How to collect and present Lessons Learnt

Eva Schiffer

The template looks straightforward and comprehensive. One comment for the sharing (both in paper and presentation form): Often people think when they have to present lessons learned, they have to do that on a high and rather abstract level which is applicable to most situations in the world. Then you often end up with the same 5 bullet points, something like: "Involve relevant stakeholders early." Everyone in the audience will think: "Yeah, I know." and not engage with the point much beyond that. I find it much more engaging and educating to tell very concrete stories about the challenge and how you dealt with it, inviting the audience to follow you, be challenged with you (while listening to you maybe thinking: "That's similar to my project." Or: "Why didn't he try this or that."), share the pain, and then tell them what you did and how it worked out (or didn't). And then you might either finish the story with: "And this is the more general lesson I learned." or "This is what I plan to do next time." or leave it to the participants to come up with more general lessons. But what they will remember and apply to improve their own work will most likely be the concrete story and not an abstract bullet point.

Paolo Brunello

In order to answer to your questions I would like to have some more context info about the workshop itself: how many people will you be dealigng with? for how long? how institutional vs practitioners-oriented will the workshop be? Can you expect to have a very flexible setting like the room we used at the KM4dev meeting @ IFAD or you are bound to the typical conference room with chairs bolted on the floor, each with a mic or with translation issues Spanish-English? Knowing about this kind of constraints would help helping ;-)

As a general comment I tend to see websurveys and templates as very useful, but risky in that they work like a sawmill: you enter trunk and you exit parquet. The richness of the details gets washed away, as also Eva was saying. So in order to compensate for such dissection of the real story, I would gently push your case providers to use their right emisphere and find metaphors, analogies and multimedia to tell their lessons learned: what would their story soundtrack sound like? If they had to draw the flow of their case on a chart hanging on the wall, how would it look like, what the bumps, the holes, the smooth slides? Could they make up and play a scene that summarises in a sharp and swift manner the typical conflict they faced and blocked them and how they overcome it? In other words, I would ask them for something closer to a performance - as theater genre - rather than a boring powerpoint. This TED video can be quite inspirational in that sense.

Paul Mundy

Two additional aspects that we need to think of when documenting project activities:

• Who is the audience of the documentation? Who are we documenting the information for? Who will be the readers or viewers?
• What are the objectives of the documentation? Why do we want to document something? How do we intend to use the information?

There are many possible audiences and objectives: internal project management, fundraising, staff training, policy advocacy, scaling up, and so on. A story that is useful for internal management purposes may look rather different from one used for fundraising or to convince policymakers that our approach is a good one. So try to determine the audience and objectives beforehand. That will help determine the type of material to collect (audio interviews, video footage, written narrative, photographs, data, some combination of these), as well as the content and tone.

Some further thoughts:

• KIS MII – Keep It Simple, Make It Interesting. Make stories simple and easy to understand, and tell them in an interesting way.
• Identifying and writing interesting stories takes time and skill. For external audiences, it may be worth investing in someone with journalistic skills to conduct interviews and write up stories, or to edit stories that staff have written into a publishable form.
• Don’t try to collect too many stories. Focus on quality rather than quantity.
• At the same time, expect a lot of wastage. A lot of stories end up being unusable for one reason or another – nothing interesting to say, lack of some vital detail, don’t fit your audience or objective, and so on.
• Be prepared to dig. The interesting material often surfaces only through a lengthy interview with a project manager or staff member.
• Good stories are hard to come by. Make multiple use of them if possible. Rework the same material for different media and different audiences if possible.

Juliana Calceido

In response to your request, I am attaching the template that we have developed for collecting lessons learned. You will see that is has just a few basic fields in an attempt to make it as simple as possible not to scare people off. We have heard over and over again that people lack time (and willingness) to fill long and complicated templates, so that's why we decided to limit the fields to the bare minimum. While your template is comprehensive and seems ideal from the content perspective, I would recommend trying to combine some of the fields to make it shorter (or at least make it look shorter!). In terms of presenting lessons, and following frequent requests for "short and sweet" from both producers and users of lessons, we really like the format used by IFC for their smart lessons (I'm attaching one for your reference).

Attachments:

• Lessons Learned template,
• Sample of Lesson Learned: "Bali Clean",
• Sample of Lesson Learned: Ukraine.

Charles Dhewa

I see you like a Soccer Coach with 17 players and want to produce a winning team. From the way the ball is passed (knowledge sharing) you are obviously going to identify goal keepers, defenders, strikers and multi-skilled players in your team.

Unless it is part of a formula such as 4-4-2 formation, ignore templates because they may prompt you to see every player as a goal keeper or striker. Focus on conversations among players to gather lessons from winning, losing. fouls, injuries, stage fright, uncomfortable gestures from spectators, etc.,. Don’t forget lessons from yellow cards, red cards, penalts (missteps, collisions, mistakes, assumptions and complete failures). There is also much to learn from match fixing (e.g., an evaluation which tells an organization what it wants to hear, among other kinds of biases).

Just as soccer coaches study matches played by other teams, your greatest lessons may be outside UNEP. Other development totems besides the UN may have the insights you want. Get lessons from other interventions in the development game. Also find out from spectators and supporters because players are often too engrossed in the game to see how else winning can be achieved. Communities that are part of each of the seventeen players you are workig with are an important source of lessons.

Although soccer players are business people while your 17 players may have a not-for-profit mindset, this analogy may open new ways of attacking your work. If you are not a soccer fan, substitute this with your favorite game. By the way, there aren't many red cards in development work!

Ian Thorpe

Until recently I was working on collecting lessons learned and good practices in UNICEF, and will next year be figuring out how we can do the same in UNDOCO on UN co-ordination work.

I don’t really have any comments on the format since it’s very similar to the one we used (perhaps due to some earlier knowledge sharing) – but other aspects to consider is how you do the collection and dissemination parts. Finding good examples within a large decentralized system can be quite challenging – even when you know that they are there.

Here are a few mechanisms we have used to identify/collect lessons:

1. Annual reporting mechanisms – adding a request for submission of examples in any annual reports that are submitted (we added a request in our country office annual reports). You might either ask people to complete the template – but alternatively you could just ask them to complete the “summary” part so you can get an idea whether the lesson is worth exploring further then follow up with the more interesting examples to complete the template.
2. Call for submissions – send out an appeal - can be good if there is a conference or publication at the end of it to attract people to submit – otherwise this can be a bit hit or miss.
3. Asking for reviews/suggestions from regional or technical advisors
4. Using e-discussions to identify potential cases – taking advantage of requests for advice, assistance or examples in communities of practice to identify potential good practices or lessons that can be further explored.
5. Repurposing media stories – stories that are written for fundraising and awareness raising and other types of publicity can be rewritten as lessons learned pieces with some additional information added IF there is some substance to them. Look for those which talk about solutions and successes and dig into them to see why they are successes and if there are any real shareable lessons in them.
6. Reviewing lessons learned sections of evaluations – evaluations often include a lot on lessons – but they are usually not documented in a way which is easily transferrable – so use evaluations to identify potential lessons – but then flesh them out according to the template to ensure key contextual information is included.
7. Organizing documentations/self reflection sessions in the office. Here we did some experimentation with designing working sessions which offices could use to carry out self-reflection on a programme, preferably involving counterparts in a one-day workshop which would identify the key elements of a lessons learned piece which could then be nicely written up. We were only just starting with this approach when I left UNICEF but perhaps my former colleagues who are on KM4Dev might be able to share the workshop outline we prepared.

As a side note on documentation, we found that if people know that these lessons will be published and promoted externally then this can actually be a great incentive to get people to share and will improve the quality (while it does mean it will take a little longer to get everyone to sign off – since in this case those involved will want to read carefully what has been written about their work). In terms of dissemination we used thematic newsletters, and we used lessons learned compendia as the main ways of distributing documented lessons, as well as putting the lessons on an online database (for internal use only unfortunately due to the IT limitations we had).

Here is a webpage with some of the compendia we did.

I’d agree with some of the previous commenters though that if you want to foster mutual learning between your programmes it is not enough to document and disseminate -what you really need to do is foster a conversation between programme managers around these examples. The brief documentation of the cases is just the starting point for the knowledge sharing. One effective way of doing this might be to organize a thematic knowledge fair or workshop where participants get a chance to discuss the project experiences in more detail. Here you might use very brief presentations to open the discussion with the main points, or even have project managers present posters or stands about their lessons as an opening session, but then organize discussion in small round-table groups using approaches such as “world café” to allow a more in depth discussion. The benefits of this are that participants get to explore the cases in more detail and ask follow up questions, as well as that those that present their cases also get feedback and new perspectives on how to improve their own programmes. The added benefit is that this helps build personal relationships between people working on similar topics across the globe who might then be more likely to share knowledge with each other in future.

UNDG organized a couple of knowledge fairs back in 2010 on country case studies with the help of the UN Staff College who developed a methodology for this (I was on the facilitation team for one of them) and might be able to give you more detailed information on the approach used for these – and I’m sure many others on this list have organized similar types of events.

Hannah Beardon

What a nice lot of suggestions and ideas. All I would like to add is that, putting myself in the position of someone on the receiving end of such a request, I think I would be more inspired (and find it easier) to respond to a really good question, rather than a form simly asking for 'lessons learned' or suchlike. If asked for lessons learned, I think my mind would go blank.... or I would dig something up which might do. But if you ask a really relevant question about aspects of my practice - for example about collecting information from diverse groups, or even - dare I say it - collecting stories of lessons learned! - i think I would have a lot more to say and more motivation to write it down. Since you already have themes, I wonder if you could formulate some interesting questions around them?

Jaap Pels

Let me try an tackle your question. I observe:

• there are some 17 projects running; there is politics / power involved.
• collecting good practice needs 'being embedded' ; you want a form to instruct a proxy collecting the stories.
• I would like to suggest to ditch forms make a blog-post the central UNEP information chunk and have contributors / readers tag information themselves. In write-shops you might want to suggest paragraph headings and a story line.
• Stakeholders should be in your network ergo manage a network. A stakeholder is not a commodity, but the 'other' end of a relationship you manage.
• By the end of a project donors should have become your best stakeholders
• 'Mainstreaming' I never understood ... perhaps lobby or public opinion needs a go in case of the environment.
• 'local management' is your target group; my question: what channels / comms means do they use? Do /can the read? Is there a language and an access issue on information made / needed?
• As for workshopping open space s the ultimate form because whoever participates takes responsibility (and one can not not participate :-) as to not participate is to participate too)
• Perhaps best your workshop is framed around your questions; see if you can get participants to think along and feel they came up with the solutions themselves how to collect and present their (!) knowledge. Just organize a couple of tables with fixed or a logical set of questions and ask one person / table to facilitate (e.g. stay at the same table) and another to report...
• Especially if you work with grow-ups teaching / lecturing / '(im)press conferencing' is not the way to go; group work! From 'talk, show, do' towards 'do, frame, talk'.

Sebastiao Ferreira

You have already received great suggestions, so I don't have much to add.

It is always possible to find better forms or structures to express lessons learned, but the performance of the form is not an internal feature, but a question of functionality to the purposes of the lessons.

The first question is: who will be the protagonists of the learning process? Which set of methods are more adequate to their cognitive preferences? Do not forget that the form will influence the method, and that the method is something in between the protagonists preferences and the knowledge product. For example: the strategic map is a form to communicate strategy, but it also has had strong influence in the way people conceive strategy and also strategize.

The second question that comes up in my mind is: Who will be the user of these lessons learned, who will take advantage of that knowledge? The needs and skills of the user will shape the form in many senses. For example: if the user is familiar with matrixes you can use this form, but most people are not used with matrixes. I know a lot of systematizations of projects whose content and forms are great for the agencies that hired the consultants, but not so great for the people who carried out the learning processes.

The third question is: which is the purpose of these lessons? Is it for challenging the current conceptions of development behind the programs? Which set of beliefs are you planning to disturb with those lessons learned? Is it for improving the role of the programs in moving forward a particular kind of social process? Is it for exploring new ways of framing the problems the programs are addressing? Is it for improving the programs, their approaches, strategies, and alliances? Is it for improving the programs the set of methods, and tools? Is it for exploring better ways of manage the programs? Or, is it about the local people involved in the programs, their culture, spirit, experience and dreams? You may say that it is for all these purposes and more. If that were the case, I would ask: where is the focus? The focus should define the organization of the lessons learned. Each one of these fields of knowledge may require particular forms to be well expressed and better used.

Amir Toister

My suggestion is to create a Wiki. I'm working with an Israeli based NGO in Kathmandu, beginning a Wiki for the same purpose. We have many volunteers in addition to the local staff and so far all the documentation is stored on a Dropbox account, but of course, it is difficult to efficiently access the knowledge and information.

I intend to run a workshop in which I will ask the current volunteers and staff how they would expect to navigate in the Wiki to quickly find what they want. It is not only to elicit information from them but also to create a relationship between them and the Wiki, turning it from "my wiki" to "their wiki", so they will want to enter their thoughts and ideas.

There should be rules for typing in the entries for the articles and IMHO using (at least informal) templates is a must, although I'm not sure if I'm going to use mediawiki templates to keep editing as simple as possible.

Lizzie Wiltshire

In terms of your workshop in February, how about using something along the lines of a World Cafe, or a Knowledge Fair? For a Knowledge Fair each of the programmes could have a ‘stall’ with a poster and resources about their successes, and the programme reps and the stakeholders can then go around the room looking at the different posters and explaining them to each other (works better if there are a few reps from each programme, as they can take turns). For the World Cafe you could place a very short description of the success on each table, and then groups could build on those successes by discussing them and writing their ideas on the paper table cloth. People could either be free to come and go from the different tables/discussions as they wanted, or you could have a few set times during the workshop when a group moves from one table on to the next. You could maybe combine the two ideas.....have a knowledge fair to kick off so that everyone gets a chance to see all the success stories, and then choose a few examples to use during the World Cafe to discuss more in depth. Perhaps during the knowledge fair people could vote for the success story they wanted to discuss in the world cafe, and you could print up a short version of those successes during the lunch break to use in the afternoon.