Participatory evaluation guide - Validating through wider data collection
Validating through data collection
The workshop sessions with project participants will bring up lots of stories of changes that have happened, and activities which have taken place. These can be triangulated and validated in a variety of ways: participants themselves can record testimonies from others, or find relevant statistics to back up their stories, or help the evaluator identify and arrange meetings with other stakeholders who can give a different perspective on the change.
Participants' data collection
Where possible, we have tried to ensure enough time with workshop participants to enable them to go back to their communities and groups and collect further information to back up their analysis, or deepen explanation of the story. In some cases we have run 5 day workshops, with
- the first two days for discussing and analysing the context and project, and planning the data collection exercise.
- days 3 and 4 for participants (in groups) to gather information and material and prepare the story or presentation
- day 5 to share the stories and collectively meta-analyse and identify the emerging themes and issues.
planning and preparation
After the sessions on context and the project, the group will have lots of ideas about what they want to say about the project (to the NGO or donors). However, they can sometimes be very interwoven and hard to pick apart. It takes some skill and some luck for clear narratives or themes to emerge from this mass of discussion and analysis. In one case I divided them into groups according to age/ role(school pupils, youth group members, youth counselors), in others where participants have come from different locations I organised them into their home groups. Then they selected the issue most interesting and relevant to them.
Once the issue has been selected, each group will need help to think about what they want to say, and where and who they need to get information from.
At some stage of the workshop, probably during the trust and atmosphere building sessions, but certainly at this point, it is good to have a discussion about audience. This is covered in 'cross-cultural communication'Participatory_evaluation_guide_-_The_approach#Effective_cross-cultural_communication but specifically here it is important for the groups to think about:
- Who are you trying to communicate to?
- What do they need to understand and why?
- What assumptions are you making about what they know about the context, issue etc?
- What do you need to explain and show them in order for them to understand?
- And what kind of language and images can help get across what you are trying to communicate?
If using film or to help with planning a story, story boarding can be used in groups or individually. See the
Interviews and follow up research by the evaluator
While the workshop participants are out collecting data and material for their stories, the facilitator/ evaluator can follow up some of the emerging findings through interviews and visits in the local area. This might also include collection of local statistics to validate, triangulate or research further some of the suggested outcomes and issues. Some of the questions and stakeholders can be identified together with the group, or selected from issues arising from group work.
This can be in the form of simple interviews, visits, focus groups, searches in local media, or discussion and research with local officials. For example, exploring changes which had taken place in Kenya, the young people mentioned a reduction in teenage pregnancy and retention of girls. They suggested I spoke to the participating teachers, who confirmed this belief. We then asked the district education department for statistics over the project life which confirmed this trend in all participating schools.