Who Measures Change? An Introduction to Participatory Monitoring and Evaluation of Communication for Social Change

Published by the Communication for Social Change Consortium, this 48-page report is an introduction to establishing a participatory monitoring and evaluation (PM&E) process to assist in the measurement of communication for social change (CFSC) initiatives. It is based on the premise that CFSC practitioners should facilitate the development of monitoring and evaluation (M&E) questions, measures and methods with those most affected and involved rather than apply predetermined objectives, indicators and techniques. The report's primary purpose is to support communication strategies based on CFSC principles when applied to critical social issues such as HIV/AIDS prevention and care. After defining CFSC and the broad purpose of monitoring and evaluation, the report explains why a participatory approach to monitoring and evaluating CFSC is useful. It goes on to discuss key PM&E principles and 'moments' or steps in establishing a PM&E process. Two 'tools' are offered to help readers learn more about and discuss: (1) potential monitoring and evaluation questions and indicators; and (2) PM&E data collection techniques.

According to the report, CFSC can be defined as a process of public and private dialogue through which people themselves define who they are, what they need and how to get what they need in order to improve their own lives. It utilises dialogue that leads to collective problem identification, decision-making and community- based implementation of solutions to development. The report explores the question - how do we know communication for social change is working?

During a December 2004 UNICEF Eastern and Southern Africa meeting on HIV/AIDS communication, the CFSC Consortium, in collaboration with an advisory committee of practitioners, was tasked with producing a manual for monitoring and assessing CFSC progress. In addition to focusing on process and outcome measures, this working group agreed upon the following principles for CFSC measurement:

• CFSC measurement tools must be community-based and participatory.
• The tools/methods must be SUM: Simple, Understandable and Measurable.
• The tools/methods must be developed with input from people from developing countries.
• M&E must be closely linked to development of the communication plans in each trial country.
• Assessment of CFSC impact should look at impact on the short-term, intermediary and long-term.
• Ultimate users want a menu of tools – not just one set of methods with no other options.
• This M&E work must build upon work done to date. The process of determining the M&E tools for CFSC must be inclusive and participatory.
• The common language of CFSC will be used whenever possible or sensible when describing the communication approach.
• Tools developed must be easy to use by groups with very few resources in resource strapped communities.
• The methods/tools/indicators developed must be immediately useful for HIV/AIDS programmes but must also be transferable to any issue requiring Communication
  for Social Change.
• Measures and evaluation need to be clear about how we assess the communication and the role of communication in helping people create self-renewing
  societies. Previous work seems to focus more on assessing social change.
• Recommended M&E approaches must be accessible to people “on the ground” and training on how to use such approaches must be detailed, yet simple to apply.

The report particularly focuses on the basic purposes of monitoring and evaluation as they relate to HIV/AIDS prevention and care programmes. The report states that "perhaps what distinguishes PM&E is its emphasis on the inclusion of a wider sphere of stakeholders in the M&E process than more conventional approaches. PM&E practitioners believe that stakeholders who are involved in development planning and implementation should also be involved in monitoring changes and determining indicators for 'success.' PM&E’s fundamental values are trust, ownership and empowerment." The following have been identified as key elements of PM & E:

• The process must be participatory, with the key stakeholders actively involved in decision making.
• The process must acknowledge and address inequities of power and voice among participating stakeholders.
• The process must be explicitly 'political.'
• The process should use multiple and varied approaches to collect and codify data.
• The process should have an action component in order to be useful to the programme’s end users.
• The process should explicitly aim to build capacity, especially evaluation capacity, so that stakeholders can control future evaluation processes.
• The process must be educational.

While the PM&E literature reviewed for this document revealed a wide variety of implementation steps, the essential can be summarised as follows:

1. Deciding to use PM&E.
2. Assembling the core PM&E team.
3. Making a PM&E plan.
4. Collecting data.
5. Synthesising, analysing and verifying the data.
6. Using PM&E results and developing Action Plans for the future.

These moments or steps are no different from conventional M&E but the key questions to remember are - What is the range of participating stakeholders in each step? And what is the degree or depth of stakeholder participation in each step?

Tool 1 provides background materials intended to foster debate and negotiation among CFSC stakeholders about what they envisage will be achieved by CSFC in
relation to HIV/AIDS and thus what might be measured. The following M&E approaches and indicator frameworks
are presented:

• Most Significant Change Technique
• Monitoring and Evaluating Networks
• Measuring Community Capacity Domain

s

• Measuring Community Participation
• Monitoring and Evaluating Health Promotion
• Measuring Social Change Communication
• HIV/AIDS Social Change Indicators
• Monitoring and Evaluating Advocacy.

PM&E practitioners use a range of different methods, depending on the nature of the programme, the context and the stakeholders. Tool 2 describes a few of the well-known PM&E data collection techniques that may be particularly useful. Considering the range of PM&E data collection techniques of potential use, some criteria should be used to decide which techniques would be most suitable to the CFSC initiative. Estrella and Gaventa (1999) suggest that PM&E data collection techniques, in general, should:

• Complement the approach and philosophy of the CFSC initiative;
• Be perceived by stakeholders or dialogue members as a way to help them address their questions and problems, not simply as information about them gathered
  by or for outsiders;
• Involve end-users in both data gathering and in analysing data;
• Match the skills and aptitudes of participants;
• Adapt to fit people's day-to-day activities and normal responsibilities;
• Provide timely information needed for decision making;
• Produce results which are reliable and, even if not quantitative, credible enough to convince others;
• Be consistent in complexity and cost to match the
  level of evaluation called for (e.g., simple and routine versus more comprehensive, major evaluations);
• Reinforce community solidarity, cooperation, communication and involvement;
• Be gender-sensitive with special efforts to include women;
• Only obtain the information needed.

Free download as a single copy. For printed 4-color copies and for multiple copies please order from the Consoritium. USD $10

Click here to order print or multiple copies online.