Skip to content

Participative, Ethnographic, Evaluative Research: Part One

February 5, 2011

‘Participation’. I have heard the word at the same time broken down – layers pealed off to reveal what some think is only a thin sheet disguising the realities of an extractive development industry– and held up to be the panacea for development programme challenges and problems. Most commonly however I have seen the word thrown around in reports and proposals, a necessary inclusion, a pre-requisite for funding.

I was not quite sure how I felt then when I found myself designing a small pilot project which had its heart the pursuit of participation. Or more specifically, participative evaluation. Observing an evaluation system which, whilst fundamentally looking to evaluate a programme which aimed to change attitudes and behaviour, found itself (understandably, and perhaps necessarily) focused on obtaining numbers from hundred of communities spread across Senegal, I found myself holding participation up as that ‘panacea’, an obvious next step, a sure fire way to ensure that community members felt like valued stakeholders, whilst simultaneously providing our evaluation department with valuable qualitative information.

And so one weekend in January, I found myself sitting down in a small classroom, in a small village, in the middle of Senegal, trying to kick start a session which would train community members to be the interviewers, the evaluators, and grappling with what this abstract concept of ‘participation’ meant in practice.

Or to be more specific, ‘Participative, Ethnographic, Evaluative, Research’ (from here on in to avoid RSI, ‘PEER’). This specific form of participative evaluation was developed by a team at Options Consultancy in the 1990’s, and has at its heart the principals of anthropological ethnographic fieldwork, in which a relationship of trust is built up between the external researcher and a community over a long period of time. When looking at sensitive issues, such as the participation of women in the community, or sexual health, this relationship of trust is crucial, and can mean the difference between snippets of useful information, and more in-depth narratives.

Working on tight time frames, this method has traditionally been unsuitable for use in NGOs programmes, which can’t afford to spend a year waiting for a researcher to sufficiently integrate themselves into a community to gain the information needed. PEER however works on the basis that this relationship of trust already exists between existing members of a social network within a community. Select members of this group are therefore trained to be the researchers, cutting out the ‘middleman’, providing the NGO or other commissioning body with valuable, and otherwise unobtainable, qualitative information which can be used to inform programme development or evaluation, and, in theory at least, giving local community members the chance to become involved the conception and evaluaion of a project.

Where as the PEER method has mostly been used to provide information which can inform the design and development of NGOs programmes, I wanted to test out how it could be integrated as part of a pre-existing evaluation system; training community members to collect information on attitudes and behaviour related to three key topics at the start, and then at the end of the programme, in order to see if any change could be reported, whilst providing a opportunity to feed any interesting findings back into the development of the programme

This came with a number of practical concerns; how will people react? What if they feel they’re being treated like spies (a valid worry, I have previously heard of Peace Corps volunteers being called this)? And in the context of research, a number of more conceptual concerns; everything I had thus far learn about evaluation worked on the basis of collecting specific, targeted information, designing questions which were highly controlled, knowing exactly what we were looking for, being able to confidently say ‘yes, this is why we asked this’. The PEER method however involves relinquishing a large amount of control; control how questions are phrased and used, indeed control over the questions themselves.

As I was repeatedly reminded however, the purpose of a pilot project is to show where things don’t work, to put it all out there and essentially, see what happens.

This relinquishing of control started with the selection of the topics; how do you choose areas to measure the impact of a programme, when the programme itself is not entirely sure where its impact will be? This was much easier in the selection of the first topic; early and forced marriage, an area where the organisation had previously shown some impact, and a natural choice for this form of research. The second was slightly trickier – I knew I wanted to focus on gender norms and the changing role of men and women, but also wondered if you could really go into a community and ask people to talk about ‘the role of men and women in the community?’, with my own preconceived idea of what this involves as its basis.

Not quite knowing what to expect, or how to focus this down further, I decided to wait and see how people react to the questions asked, and that this would be part of the learning process. Finally, I was ready pick the third topic, and had in my mind the issue of decision making surrounding health; another area where I knew the organisation expected to have some impact, yet in what specific forms it was difficult to assess through traditional quantitative methods.

The involvement of ‘participation’ however within this question selection had been thin, and it seemed that it was time to practice what I preached, and ask the local programme staff, those living in, and near the community, what should be the focus of the evaluation. ‘Violence against women’ was the immediate response. In any evaluation, the involvement of local staff and stakeholders naturally raises a number of tricky questions, including, ‘how do you balance the evaluation needs of the organisation, and most usually, the donor, with the observations of field staff?’ Not having a background in the area of violence against women, and aware of the many ethical difficulties of conducting research on this issue, even trickier questions were raised; could raising such a complex and sensitive issue jeopardise the whole process if we face a wall of silence? What if I am inadvertently putting researchers themselves in an uncomfortable position, asking them to talk about subjects which may just be too close to home?

Several hours of reading up on the ‘ethics of researching violence against women’, by no means a solution, but it seemed, a basic first step,  and some discussion with colleagues on how to address some of these ethical questions, it was time to dive in, and respect the input and knowledge of those I originally consulted.

One week later, the training had begun, and, standing in front of a room of strangers leading an ice breaker of ‘heads, shoulders, knees and toes’, I took the first few steps in learning about the on the ground challenges involved in a participatory training session, and learned a valuable lesson in the importance of humility in facilitating this process.

Part two can be found here

Advertisements
2 Comments leave one →
  1. Dad permalink
    February 5, 2011 5:39 pm

    I didn’t know you knew ‘head, shoulders, knees and toes’ and can’t imagine what a spectacle this must have been. You certainly know how to keep them guessing- ‘what is she talking about today?’ Or may be it was the spectacle of a red haired white woman doing calisthetics that held their attention as you worked your way into this sensitive area of family life? Look forward to hearing how the training session and field work went in your next report. Dadx

Trackbacks

  1. Participation as empowerment? « pas à pas

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: