6.1 An Overview of Gathering,
Managing and Communicating Information
Knowing the Journey Data Will Take
On this journey they are gradually collated and analysed as the data move
from field sites or different project staff and partner organisations
to be centrally available for management decisions and reports. The journey
involves a transformation from data to information and knowledge that
is the basis of decisions. Data are the raw material that has no
meaning yet. Information involves adding meaning by synthesising
and analysing it. Knowledge emerges when the information is related
back to a concrete situation in order to establish explanations and lessons
for decisions. Many rural development projects have much data lying around,
less information, little knowledge and hence very little use of the original
data for decision making (see Box 6-1). To avoid this
problem, plan not only how you will gather data but also how you will
transform the data into valuable knowledge.
6-1. Data and yet no information in Uganda
one project in Uganda, field extension staff had kept monthly records
for seven years on their work with farmers to establish sustainable
livelihood activities, such as planting woodlots, beekeeping, using
fuel-efficient stoves and implementing soil conservation measures.
There was literally a room full of monthly reports. However, no
system had been developed for collating this information and turning
it into insights about adoption rates, reasons for differences between
villages or differing success rates of particular extension staff.
When analysis of the data was attempted, it proved to be impossible
because the data was unreliable and very difficult to compare and
collate between different project areas. This problem typically
arises when the focus is on data collection rather than knowledge
6-1 shows how data travel. Table 6-1 lists questions that need to
be considered for each part of the journey. For each performance question
and indicator, the journey will be different in terms of the choice of
methods, frequency and responsibilities. Irrespective of the journey,
be sure that the information you are collecting is helping you answer
your performance questions (see Section
6-1. The journey data take
6-1. Preparing the journey for your data
Questions to Answer
a sample be necessary? If yes, how will it be taken in order to
be representative of the project’s primary stakeholders? If no,
where can you get the information?
are you going to find your information: by measuring, interviewing
individuals, group discussions, observing?
will use which formats to write, visualise, photograph or take video
of data and impressions?
will data (raw and analysed) be stored, how and by whom? Who will
will use what methods to group data into a logically ordered overview?
will examine the data using what method to give them meaning and
synthesise them into a coherent explanation of what happened and
what needs to now be undertaken?
feedback and dissemination
what stages and using what means will information be shared with
project and partner staff, primary stakeholders, steering committees
and funding agencies?
Considerations When Choosing Your Method
Before choosing your method, be clear about three methodological aspects:
- the difference
and overlap between methods for qualitative and quantitative information;
- the implications
of working with individual or group-based methods;
makes a method participatory – or not.
Several steps need to be followed to select the most appropriate method(s)
(see 6.2.2 for more details):
that you are completely clear about what information you need collected,
collated, analysed or fed back, for which you are seeking a method.
that another group, person or organisation is not already collecting
the data. Check, where possible, how the information was collected to
see if it is reliable enough for your needs.
- Be clear
about how accurate you need to be.
- Does the
information relate to a specialist area? If so, seek specialist advice
or documentation before proceeding with the method selection.
- Be clear
about the task that needs to be accomplished, and whether this concerns
qualitative and/or quantitative information. Consider whether a method
is needed to collect, collate, analyse, synthesise or disseminate information.
the extent to which the data gathering or analysis process is to be
participatory, and therefore whether you need to work with individuals,
groups or a combination.
if your data-collection coverage is to be sampled or comprehensive.
If working with a sample, decide on your sample size, clarify the "sampling
frame" and select your sample (see D.1).
- Do you
have several methodological options or is there only one? List your
method options and make an initial selection. If using a sequence of
methods, check that the methods complement each other.
- List your
methods and make an initial selection.
- When you
think you’ve got the right method for the task at hand, consider if
it is: feasible, appropriate, valid, reliable, relevant, sensitive,
cost-effective and timely.
your method, with a small number of participants who are similar to
those from whom information is going to be sought. Adjust your method
based on recommendations from the test run.
the frequency of use.
Gathering, Collating and Storing Information
When preparing for data gathering, do not forget to:
carefully how to select interviewers and facilitators.
how to distribute the tasks of collection and analysis among different
people and what is needed to limit errors.
that those using the methods are comfortable with them.
clarity of language.
the practicalities of each method, such as materials needed.
by considering possible causes of sampling errors and non-sampling errors.
Non-sampling errors are particularly critical. These can occur due to
interviewer bias, inadequacy of methods, processing errors and non-response
bias (see 6.3.1).
data from time to time. Spot checks are important at the beginning of
any project – if you are using existing data sets – by looking at where
data come from, who has collected information and the methods and standards
they used. Also check data collection when using a new method or when
working with new fieldworkers, new implementing partners, new staff, etc..
Data can be suspicious if you notice overly precise data (like perfect
matches between targets and actual realised activities), sudden large
changes in data, and data gaps.
bit of information, define how it will be recorded. Practise with the
people doing the recording before setting out to collect data.
of collating (or aggregating) information often gets lost in the gap between
data collection and analysis. It requires some attention as it can greatly
facilitate analysis if undertaken well and can introduce error if done
poorly. Collation is needed when you are scaling up your information from
a smaller unit of analysis to a larger one or when information has been
collected from different sources with different methods. The collation
of qualitative data requires special care and analytical skills.
and quantitative data analysis are both critical for making use of M&E
data but are also quite distinct processes. The Guide focuses on aspects
of qualitative data analysis as statistical procedures fall outside its
scope. Refer to Section 8 for
many ideas on how to encourage reflective meetings and analytical reporting
in addition to the ideas in 6.4.2.
how to organise the storage of M&E information, consider these four
questions(also see 7.5):
- What information
needs to be stored?
- Who needs
access to the information and when?
- What type
of information needs to be stored – hard copies or data that can be
computerised and accessed centrally?
assess what information you need to keep and what can be discarded.
Considering Communication of M&E Results
findings have many potential audiences: funding agencies, steering committees,
cooperating institution, project and implementing partner staff, and primary
stakeholders. The main purpose of communicating findings is to ensure
accountability and motivate stakeholders to action. Draft M&E findings
need to be discussed with implementing partners and primary stakeholders
in order to get feedback on accuracy, reach joint conclusions and agree
on next steps. Final findings can then be passed to the relevant organisations
for accountability and action.
carefully how you will communicate your M&E findings. Reach agreement
with project stakeholders on who needs to receive what kinds of M&E
information. Remember to include accountability, advocacy and action-oriented
audiences and to agree on the information (content and form) they need.
for communication as part of your M&E system from the outset. Do not
hope or expect that someone else in the project will communicate M&E
findings. As part of this, invest in good communication, not only in producing
effective outputs but also in project-based capacities for communication.
key communication task is to ensure that your findings are correct. Workshops
and meetings are critical events to seek feedback and plan action.
planning to present M&E information for feedback, consider these practical
clarity of message for specific audiences.
on the frequency for communicating information.
timeliness. When do you need to get feedback to still be useful for
location. Where will people feel at ease?
media to communicate findings. Written reporting is most known and ranges
from formal progress reports, to special studies, to informal briefs in
the form of memorandums highlighting a current issue. M&E findings
can often be communicated more effectively verbally than by other means.
Speaking directly with a target audience provides a quicker and more flexible
way to convey your message. Also use visual displays, such as graphs or
charts showing trends or maps, to convey summaries of what is happening.
6.2 Deciding Which Methods to
What Are Methods?
is an established and systematic way of carrying out a particular task.
Agronomists have methods for measuring crop yield. Economists have methods
for calculating return on investment. Anthropologists have methods for
looking at household decision-making patterns. Accountants have methods
for budgeting and reporting on project funds. And managers and facilitators
have methods for helping groups to make decisions.
makes use of a wide range of methods for gathering, analysing, storing
and presenting information. In your M&E activities, you are likely
to use established research methods from the biophysical and social sciences,
as well as from a growing collection of participatory methods (see Box
6-2). Sometimes the information you require will make it necessary
to adapt an existing method or develop an entirely new method.
6-2. Matching methods to needs
IFAD-supported agricultural development project in China used crop
development models to make predictions on the development of 14
crops, including the impact of staple and specialty crops – such
as pearl sorghum and ginger – on farm-level production and income
generation. These models were calculated with the help of the FARMOD
modelling software developed by FAO and the World Bank. These estimates
could be used as a base with which to compare actual results gathered
through data-collection methods.
India, a method for the self-evaluation of women’s credit "self-help
groups" was developed for periodic monitoring of specific indicators.
Because many of the women are illiterate, a series of pictures was
used to represent indicators and a colour-coding system was developed
to represent levels of evaluation. This method was used in groups
and allowed for full participation of all the members.
carrying out M&E, it is often necessary to combine a series of methods
(see Box 6-3). For example, a participatory rural
appraisal (PRA) process used to find out how primary stakeholders are
benefiting from a project might combine some 15 or more different methods
ranging from transect walks to matrix ranking and focus group discussions.
Likewise, a household survey or annual project review meeting would combine
a series of interviewing, discussion and facilitation methods. The combination
of a series of methods in a structured way is often referred to as a methodology.
For example, you have a methodology for a workshop or a methodology for
a baseline survey.
Box 6-3. Diverse methods for sustainability
monitoring in the Karnataka Rural Water Supply and Sanitation Project,
village-based sustainability monitoring process was developed to
understand what issues could potentially adversely affect the sustainability
of water and sanitation services in India. A set of nine questionnaires
was developed to be used in visits to 15 villages, with the following
topics: village socio-economic profile; technical: water supply
(asset condition and profile); technical: sanitation (drainage,
soak-pits and dustbins); technical: sanitation (household latrines);
financial: costs, tariff, billing and collection; institutional:
village water and sanitation committee (VWSC) – composition, functions
and effectiveness; household: facts, perception of demand met; social:
participation by women and poor; and tap stand monitoring.
and Data Collection
starting the data collection, a one-day preparatory workshop was
held for the teams to brainstorm about the concept and the methods.
A variety of methods were used in order to answer the questionnaires:
direct observations, general meetings, focus group discussions,
household surveys, and observations and interviews of villagers
while collecting water at the public tap stands.
the fieldwork, all the data collected through the questionnaires
and scores of the 71 indicators were converted into a sustainability
index for each village. The analysis revealed that nine out of the
15 villages visited fell into the "likely to be sustainable"
category (60% with a score above 0.65), five into the "uncertain"
category (33% between 0.50 and 0.64) and one in the "unlikely"
category (below 0.50).
Types of Methods
D provides a description of 34 different methods commonly used for
M&E and, in particular, participatory M&E. They have been grouped
- core M&E
methods (such as stakeholder analysis and questionnaires);
methods for groups (such as brainstorming and role plays);
for spatially-distributed information (such as maps and transects);
for time-based patterns of change (such as diaries and photographs);
for analysing relationships and linkages (such as impact flow diagrams
and problem trees);
for ranking and prioritising (such as matrices).
probably also need to draw on other specialised methods related to specific
technical fields, which are clustered under biophysical measurements (Method
5) and cost-benefit analysis (Method 7) in Annex
on specific technical expertise when developing a detailed M&E plan,
you can ensure the inclusion of appropriate specialist methods.
your methods, first consider three important aspects:
versus qualitative methods (see Table 6-2);
versus group-based methods (see Table 6-2);
- the extent
to which a method can be participatory.
Table 6-2. Examples of multi-purpose M&E methods
studies, brainstorming, focus groups, SWOT, drama and role plays,
maps, transects, GIS, historical trends/ timelines, seasonal calendars,
rich pictures, visioning, flow diagrams, well-being ranking
group technique, maps, transects, historical trends/timelines,
seasonal calendars, flow diagrams, matrix scoring and ranking
interviews, case studies, maps, transects, diaries, historical
trends/timelines, seasonal calendars, flow diagrams
measurements, structured questionnaires, maps, transects, GIS,
diaries, flow diagrams
and Qualitative Methods
methods directly measure the status or change of a specific variable,
for example, changes in crop yield, kilometres of road built or hours
women spend fetching water. Quantitative methods provide direct numerical
methods gather information by asking people to explain what they have
observed, do, believe or feel. The output from qualitative methods is
in M&E reports tends to be based on numbers. Quantitative data are
clear and precise and are often considered to be more scientifically verifiable.
You will always need this kind of information. However, for some performance
questions you will need to complement it by asking people about their
experiences and opinions.
to use a method to produce or analyse qualitative or quantitative data
(see Box 6-4) depends not only on the type of information
you are seeking but also on the capacities and resources you have available,
how the information will be used and how precise data need to be (see
the difference between quantitative and qualitative methods is not absolute.
Much qualitative information can be quantified. For example, opinions
can be clustered into groups and then counted, thereby becoming quantitative.
Note, however, that you can never make quantitative information more qualitative.
You cannot extract an opinion from a number.
Box 6-4. Using methods to produce quantitative
or qualitative data
for quantitative data. They need to produce data that are easily
represented as numbers, answering questions such as "How much…?",
"How many…?", and "How frequent …?" Quantitative
data generally require formal measurements of variables such as
income, production or population densities.
for qualitative data. They produce data that are not easily
summarised in numerical form, broadly answering the "how"
and "why" through, for instance, meetings, interviews
or general observations. Qualitative data are more appropriate for
understanding people’s attitudes or behaviours, beliefs, opinions,
experiences and priorities. Qualitative data include answers to
questions like "Why do you think this happened?" and "How
do you think this will affect you?"
Box 6-5. Considering the pros and cons
of qualitative and quantitative studies 2
study focusing on the community’s acceptability of immunisation
was carried out in Somalia, as mothers did not seem to want to take
their children to be immunised.
quantitative survey could have found out: how many mothers
accept immunisation, how many do not and whether this is related
statistically to their socio-economic status, education, age, number
of children, distance from the clinic, income, clan, etc. This information
might be useful for programme planning if the social or physical
factors that were found to influence the mothers could be changed.
a qualitative survey was used instead. It found out why mothers
do or do not take their children to be immunised. It looked at their
experience with immunisation and how that affects their behaviour.
The study showed that the way mothers were treated in clinics put
them off. For example, they were not given enough information and
were scared when their children suffered from fevers after vaccination.
They also thought that diseases were caused by bad spirits and,
therefore, could not be prevented by vaccination.
this study, it was possible to change the way clinics were run and
how staff was trained, and it was easier to explain to mothers why
immunisation is important.
Individual- or Group-Based Methods
the M&E process – from design, to data collection and analysis – you
can choose to use methods to consult with groups or with individuals (see
Table 6-3). Working with individuals can give you
more detailed information but it will only give an overview after analysing
data from a set of individuals. A group-based method will elicit a more
collective perspective – with areas of consensus and divergence – while
personal details and perspectives are less likely to emerge. Groups ask
more of the facilitator and the quality of discussions depends on the
size of the group and how comfortable people are with each other and the
topic at hand. Annex D includes one cluster of methods that are particularly
suited for group discussions. However, many other methods in Annex D can
also be used in a group context (see Table 6-2).
people involved at any one M&E event, the greater the importance of
good facilitation and planning. The facilitator’s skill will largely determine
whether a method is used successfully in a group. Good facilitators will
provide suggestions, probe, encourage, redirect and also take notes. They
also help manage conflicts by encouraging people to listen to and understand
each other’s perspectives.
6-3. Pros and cons of working with individuals and groups
Manage the discussion more easily
Can get detailed information
Generate data that can usually be structured in a way that makes
statistical analysis possible
Consume more time if you want data from many individuals
Cannot be used to generate consensus
Do not allow cost-effective feedback
with a group
Generate new learning in some participants, as information may be
shared that normally is not
With careful planning, can allow for marginal voices to be heard
Can show where divergence and convergence of opinions lie
Can cause problems in terms of data validity, as individuals may
be influenced by group dynamics or composition
Cannot (usually) deal with sensitive information
Require a facilitator able to deal with group dynamics
Require careful thought about group composition to adequately represent
the voices you want to hear
a Method Participatory
are keen to involve primary stakeholders more in M&E. They commonly
consider that collecting data from local people using so-called participatory
methods is sufficient. Imagine the following scenario. The M&E staff
of a project goes to a group of farms to understand if soil nutrient flows
have changed as a result of farmer training on soil conservation. They
meet the farmers and ask them to sketch maps showing where nutrients enter
the farms, how they are used and how they leave, and in particular showing
what has changed after soil conservation measures were adopted. The mapping
process lasts about two hours, after which the team goes back to the M&E
office with the sketched maps to synthesise and analyse the data for a
report to the director. At some point, the report is copied and sent to
the village. Can you call this mapping process participatory?
in M&E is often limited to working with primary stakeholders as information
sources, rather than as joint users of information and therefore potential
analysts and co-designers of methods. If you have selected the method
and use it to get information from people, then you are involved not in
a participatory process but in an extractive one. This is fine – unless
you are aiming for participatory M&E. In which case, you would involve
other stakeholders in choosing and using methods.
think there is a set of so-called "participatory" M&E methods,
but this is not the case. A method is not inherently participatory or
not participatory. Many of the methods useful for M&E can be used
in either a participatory or non-participatory way. The participatory
impact comes with the way a method is used and who helped select it. The
use of a technical method for testing water quality, for example, can
become participatory if the community is involved in deciding what aspects
of water quality to measure, collecting the data and reviewing the results.
On the other hand, if a group is directed to produce a map of the area,
there is little discussion, and the map disappears into the project office
forever, then this cannot be called participatory mapping. See 2.6
for general considerations for participatory M&E.
that the selection and use of methods is participatory, consider these
what aspect of the M&E methods is participation important? In
selection or design of the method, in applying it for data collection
or for analysis?
should ideally be involved in the task at hand? Who needs to help
select, design or use the method? Ideally, those who are to use it for
collecting or analysis should be involved in selection/design. This
can include staff of implementing partners, project staff, primary stakeholders
wants to be involved in what? Not everyone has the time or
inclination to participate. This is not a problem, as full participation
is neither practical nor possible. Instead, you need to ask those you
would like to involve if they are able and interested.
is needed for effective participation? Self-confidence is needed
before effective participation is possible. Therefore you need to create
the conditions for people to feel free in helping define methods, in
testing and adjusting them, in collecting data, etc. This can include
providing training or follow-up mentoring, finding the right time and
place, offering childcare support, etc.
Selecting Your Methods
the most appropriate methods for the task at hand, the steps below can
give some guidance.
clear about what you need to know. Section
5 discusses the process of deciding what you want to monitor and
evaluate. Before you start with method selection, confirm with those
involved that everyone is clear on what information needs to be sought.
that another group, person or organisation is not already collecting
the data. Before investing in method selection for data gathering
and analysis, find out if the information you are seeking is already
available and from where (see Table 6.3). Government
agencies, universities and research organisations will often have data
that can contribute to the project’s information needs. Start by asking
whether there are reporting mechanisms in the villages, local towns,
district capitals, etc. for information you might need, such as population,
disease incidence, tax collection and so on. The methods employed will
be many and varied, ranging from national statistical and census methods
to specific research methods. You might find it helpful to make an inventory
of existing information collection, as in an IFAD-supported project
in Zambia (see Table 6-4).
possible, how the information was collected to see if it is reliable enough
for your needs. In some situations it may be possible to modify data gathering
by other agencies to better support the M&E work of the project. However,
if you think the data quality cannot be improved or if they are too difficult
to access, then you will need to consider collecting the data yourself.
6-4. Part of an inventory of information useful for the District Development
Project that is already being compiled in Zambia3
of Information Collected
Does It Go After Collection?
water affairs, education, MoH (min. of health)
Planning for new water points and maintenance
authorities, NGOs, water affairs, UNICEF, MoH
department, local authorities, MAFF (min. of agriculture, food
Planning, e.g., access and maintenance
(min. of local government and housing), MAFF, MoT (min. of tourism)
camps, hospitals, schools, industries, trading centres, banks,
(central bureau of statistics), local authorities, sector departments
Planning for services provision
Planning for new investment
sector departments/ministries, donors, MoFED (min. of finance
and economic development)
Crop production potential
MoFED, CBS, FRA (food reserve agency), local authorities
teachers, CBS, inspectorates (district), zone coordinators
Planning purposes, e.g., upgrading, expansion, materials procurement
curriculum development centre, MoH, local authorities
authorities, hospitals, CBS
Birth and mortality rates
Population growth rate
Planning, e.g., provision of social services
general, MoFED, CBS, MoH, MoLGandH
clear about how accurate you need to be.
Higher accuracy is always more desirable than lower accuracy. However,
in some cases you may not need precise figures or detailed opinions
based on a representative sample, but only a general impression. For
example, you can choose to do a series of 50 measurements on farmers’
fields to measure exact productivity. But you might only need to know
if most farmers are satisfied with their yields, for which discussion
with several farmer leaders might be sufficient.
the information relate to a specialist area? If so, seek specialist
advice or detailed documentation before proceeding with the method selection.
This is the case, for example, for cost-benefit analysis and geographic
information system mapping (see Methods 7 and 19 in Annex
D). They require expert input in order to assess if they are worthwhile
for the project to use.
clear about the task that needs to be accomplished and whether it concerns
qualitative and/or quantitative information. Consider whether a
method is needed to collect, collate, analyse, synthesise or disseminate
information. Does the performance question or indicator for which you
are seeking a method require quantitative, qualitative or both types
of information? Think about whether you need individual or group opinions.
Also consider how the people involved prefer and are able to communicate,
as this determines the choice of medium: written, oral, visual and/or
dramatic. Some methods are based on diagrams, while others focus on
the extent to which the data gathering or analysis process is to be
participatory and, therefore, whether you need to work with individuals,
groups or a combination. Different stakeholders can be involved
in data gathering and analysis of information to varying degrees. Be
clear about why you are seeking more participation (see Box
6-6). Is it for consistency in processing or for shared analysis?
This will affect the choice of method. The extent of participation will
also influence the suitability of certain methods. For example, a cost-benefit
analysis is not suited for just anyone, but for someone with an economic
background. If you are developing an M&E system that micro-credit
groups are to implement and manage, then questionnaires will only be
suitable if they design this themselves and are confident about analysing
6-6. When participatory M&E is the incentive needed to keep
the data journey moving 4
many CARE offices, there is often a physical and temporal gap between
data collection and data analysis. Those collecting data are often
not involved in analysing it. Analysis often happens months after
the data are collected. Often data are not analysed at all. One
M&E staff member in CARE joked that when he started his job,
there was a huge container of paper outside his office that one
day simply disappeared. He was indicating that unanalysed data can
easily disappear without being missed.
Bangladesh, CARE project staff tried to meet this challenge by introducing
participatory methods into their project monitoring systems. Shifting
their monitoring activities from CARE headquarters to the field
level grew out of concern that data analysis was not done by those
who collected the information nor who were involved in the day-to-day
running of the project. Also, it took so long for headquarters-based
staff to receive monitoring forms, enter data, send forms back to
the field for corrections and so on that data processing sometimes
took over a year.
participatory M&E was introduced to:
Increase the validity of monitoring data by having field trainers
and project participants involved in analysis;
Increase the quality of data by helping participants become aware
of why they are being asked certain questions.
project team has now prepared forms that are one-page pictorial
summaries of production and input data, which will be used with
farmers. This data will then be entered and analysed at the "thana"
and district levels. Composite reports will then be sent to headquarters,
where they will be compiled and analysed for the project as a whole.
6-7. Random sampling within a non-random sample5
a total of nine villages, nine to ten households were selected randomly
from four different income categories from each village. The nine
villages consisted of three villages from clusters in three different
geographical areas. In each cluster, villages were selected on the
basis of the length of the project in the area (i.e., one, three
or five years). This sampling allowed for two types of comparisons
to be made. A comparison was made based on the length of the project’s
presence in the village and one was made across clusters (geographical/topographical
you have several methodological options or is there only one?
Armed with all these details about how you hope to find information,
ask yourselves if you actually have any options. Sometimes the type
of information you are seeking can be found clearly only in one way.
For example, knowing how many turtles have laid eggs on breeding beaches
will require you to go and look. However, it is more likely that you
will have several options.
your method options and make an initial selection. Once you know
what the method needs to do, then it is time to list all options and
choose. Table 6-5 provides one way to help you organise your thinking
for this step.
6-5. Helping you match methods for performance questions and indicators
Question / Indicator
in Gathering Data
on Possible Methods
this from your M&E matrix (see Section
5 and Annex
degree of participation, qualitative, quantitative, who is to
do it, etc.
particular potential problems and key advantages.
of your method will depend largely on the type of information needed,
the skills of those involved and the degree of precision needed. Also
make sure that methods complement each other to provide the information
you are seeking and that they allow you to crosscheck information. For
example, a forestry resource management plan may involve GIS maps (Method
19), resource mapping (Method 17) and transects (Method 18) to gather
information on the forest resources, an analysis of historical trends
to understand changes in forest use and ownership, an institutional analysis
diagram (Method 27) to help with stakeholder analysis and various discussion
methods (Methods 11 to 16) to understand local priorities and dynamics.
in your selection process is ensuring appropriateness. Table
6-6 provides an example of the appropriateness of different soil-erosion
assessment methods for different audiences. Especially in the case of
participatory monitoring, methods should be selected so they can eventually
be incorporated into everyone’s everyday activities, as few people are
likely to be remunerated for the effort involved. Methods might need to
be created after negotiations about appropriateness (see Box
6-8). Where possible, the information collection, analysis and the
use of the results should be undertaken by the same people, who should
understand the method(s) and agree that they are appropriate.
Table 6-6. Appropriateness of soil-erosion assessment
methods for different stakeholder groups 6
(rills, turbidity of run-off water, etc.)
in the ground
6-8. Negotiating appropriate methods in Brazil 7
Brazil, the farmers, NGO staff, union representatives and university
academics were deciding which method could assess "the percentage
of vegetation cover" (one of the chosen indicators for monitoring
their agroforestry activity). First, the academics suggested using
a wooden frame (with four quadrats of about one square metre in
total), to be placed on the ground in several sites within the agroforestry
plot) and visually estimating the surface area covered by vegetation.
They also suggested a form to fill in the percentages. While the
wooden frame was acceptable, the farmers thought the form would
be too complicated. The academics then suggested a form with pre-drawn
quadrats that the farmer could shade to depict the area under vegetation.
Again, it was rejected as too alien to the farmers’ way of registering,
as they are reluctant to use pen and paper. Finally, they all agreed
on the use of wooden sticks or rulers, on which the farmer scratches
a mark to indicate the estimated percentage of vegetation cover
in terms of a certain segment of the ruler. Each farmer uses a new
stick for each measuring event. When the farmers meet for the agroforestry
project, they bring their rulers, register the measurements on paper,
and discuss the findings and the significance for their plots.
might well debate the accuracy of a scratch mark on a wooden stick
compared with written percentages on a piece of paper. However,
if the paper-based method had been imposed, the reliability of the
information would probably have been low because the farmers were
reluctant to use this approach. In this case, participation probably
ensured a more realistic version of "rigorous" data collection.
this checklist to see if you have the right method for the task at hand.8
Do you have the right skills and equipment for the method? Can the method
realistically help cover the intended questions/indicators? Do you have
enough time? Can you cover the geographic area adequately? What is the
distance between participants and what are the language requirements?
Are sufficient technical support and training provided?
Does the method suit the conditions of the project? Does everyone involved
agree that the method is appropriate and do they understand it? Is the
unit of analysis appropriate for the method?
Do the people who are to use the information believe the method is valid,
i.e., able to assess the desired indicator with enough accuracy?
Will the method work when needed? Is the error that will occur acceptable?
Are you using different methods to verify the information collected,
rather than using only one particular method so risking distorted information?
Does the method produce the information required or is it actually assessing
something similar but, in fact, basically different? Does the method
complement the basic philosophy and approach of the project?
Is it able to pick up data variations sufficiently? Can it be adapted
to changing conditions without excessive loss of reliability?
effectiveness. Have sufficient financial resources been allocated?
Will the method produce useful information at relatively low cost –
or is there a cheaper alternative that provides information that is
Is there an acceptable level of delay between information collection,
analysis and use? Do the methods use the least amount of time possible
outside of everyday work? Have you looked for ways to incorporate the
use of the methods into other daily tasks?
your method. You should pre-test all M&E methods to make sure
they are feasible and will give you the desired kind of information.
Pre-testing is particularly critical prior to a major data-gathering
exercise. It involves a trial run with a small number of participants
who are similar to those from whom information is to be sought. Check
that the questions are clear and see how long the method takes per
person or group. Adjust your method based on the outcome of the test
run. You might need to organise additional training if the method
seems to require more skills than those possessed by the people who
are to use it.
the frequency of use. Monitoring implies repeated use of a method
to make comparisons, for example, returning to a map (Method 17) every
six months to update the information or holding a focus group (Method
12) to see if views have changed. Methods need to be consistently
applied at each monitoring moment so that information is not distorted,
comparisons are possible and findings are reliable.
6.3 Gathering Data from the Field
Preparing and Planning for Data Collection
and pre-testing the method – but before starting the data collection –
you will need to make the final preparations. Consider what you might
need to do to limit common problems in the field.
carefully how to select interviewers and facilitators. Two types of
fieldworkers will be needed: interviewers to collect data and facilitators
to conduct group-based discussions and analysis. Interviewing and facilitating
are two sets of complementary skills. Consider whether the following factors
may influence the quality of interviews and discussions: age, gender,
background and position in the community, educational level, socio-economic
level, personality and attitude, physical health, language, religion and
cultural customs. These factors may impair or enhance an interviewer or
facilitator’s capacity to understand certain topics or be acceptable to
whomever he/she is meeting. Select those people who fit best with the
task at hand and the stakeholders with whom they will interact.
how to distribute the tasks of collection and analysis among different
people and what is needed to limit errors. The number of people involved
in each stage of the data journey will affect the consistency and accuracy
of data. The greater the number of people involved, not only the more
organisation is needed but also the greater the risk of data inaccuracies
and inconsistencies. Plan how you will ensure that fieldworkers achieve
consistent quality of data collection/facilitation and how data will be
verified (see 6.3.2).
that those using the methods are comfortable with them. Each method
should be pre-tested and practised by individuals who are to apply them.
Facilitation techniques need to be mastered by those who will interact
with stakeholders to collect and analyse information. This means understanding
and practising facilitation techniques but also having the skills to design
methods jointly with stakeholders. A training session on methods needs
to cover the purpose of each method and of data collection and analysis,
improve the specific skills for working with groups and doing good interviewing,
and teach ways to record information.
clarity of language. Ideally, field workers either speak the relevant
language or are accompanied by a trusted interpreter. If working through
translation, spend time getting the translations right with native speakers
and, if possible, train the translators in the selected M&E methods.
A list of clear translations needs to be prepared before the fieldwork
starts. One way of ensuring that an unusual method, such as matrix scoring
(Method 32, Annex D), is
translated correctly is by having one native speaker translate it and
then asking another person to translate it back to the original language.
Then the two versions can be discussed with the data-collectors to be
sure they understand and can comment upon the nuances involved.
each method. Each method will require its own preparations (see Box
6-9). Be sure to organise materials, including sufficient backups
of the measuring and recording instruments (pencils or pens for filling
in forms or questionnaires, notebooks in which to write, markers for flip
charts, batteries for a laptop computer or tape recorder, etc.). Carefully
plan the formats needed to record information (see 6.3.3)
Box 6-9. Examples of methods and their
checking of forms by a professional to be sure that questions
are unbiased and formulated properly, enumerator training to
ensure they understand the questions and record accurately,
availability of enough copies of the questionnaire, provision
of several writing instruments (and tape recorder if necessary).
measurement: forms for recording, training in the accurate
use of the measuring instrument, spare instruments and spare
parts if budget allows.
plays: effective training for good facilitation and drawing
conclusions together with participants, (video) camera, notebook,
flip chart, tape recorder, pens.
mapping, flow diagrams, matrices: training on facilitation
and explanation of its purpose, (extra) paper, coloured pens,
notebook for own notes.
methods: training in facilitation techniques, flip chart(s)
and coloured pens, notebook.
Ensuring Reliability of Information
of information is about consistency. To increase the reliability of information,
stop to consider possible causes of inconsistency. Errors creep into the
system when, for example, field staff document answers inaccurately, selected
respondents are not the best information sources, field staff are unclear
about the purpose of information gathering, etc. Two basic types of data
errors are "sampling errors" and "non-sampling errors".
sampling error occurs when you have chosen the wrong sample (see 6.2.3
and D.1, Annex D). It
is the difference between an estimate derived from a sample survey and
the value that would result if a census of the whole population were taken.
For example, if a sample has a response rate of 30%, the sample error
estimates how accurately the sample has estimated the 30% of the population
that it supposedly represents. Sampling errors arise when the information
you have collected does not accurately represent the target population.
Casley and Kumar (1988: 81) list the types of households that could be
missed when compiling a sample, resulting in data biases: remote or inaccessible
households, those with frequently absent members (e.g., migrant labourers),
newly created single-person households and ethnic minorities (as they
are often marginalised within a village). See D1 for information on how
to select a sample. Sampling errors do not occur in a census, for example
if you ask all the micro-credit groups the same questions. Because you
have involved all of them, you will only have non-sampling errors.
most common and diverse types of errors are the non-sampling errors. Knowing
the possible causes of systematic non-sampling errors can help you limit
bias. An interviewer can unfairly influence the way a respondent
answers questions. This may occur if the interviewer or facilitator
is too friendly, aloof or prompts the respondent. Fieldworkers need
to have adequate capacities but also the right incentives. This can
also be caused by a management culture that discourages the reporting
of problems such as low levels of implementation (see Section 7 for
more on incentives).
methods. Causes include: complicated collection procedures, inappropriate
formats, ambiguous questions, mismatch of questions and method, etc.
errors. These can arise through miscoding, incorrect data entry,
incorrect computer programming and inadequate checking.
bias. If a significant number of people do not respond to a certain
question, then results may be biased because the characteristics of
non-respondents may differ from those who have responded. Some questions
may be difficult to understand for certain people.
error can occur at any stage of a sample survey or census, and unlike
sampling error, it is not generally easy to measure. The non-sampling
errors are difficult to measure due to the diversity of sources (the interviewers,
respondents, coders, data entry operator, etc.).
inaccuracies can have more than one source of error. For example, in a
micro-credit project in India, the implementing partners felt that data
collected were inaccurate due in part to a burdensome and cumbersome process.
The NGOs also questioned the capacity level of local groups to fill out
the lengthy monitoring formats accurately. Furthermore, there was very
high turnover of grassroots workers, primarily due to very low salaries
paid under the programme, so consistency of data collection was bound
to suffer. The NGOs feared that if primary data were not accurate, then
errors would multiply as the information from the different groups and
staff was collated into larger figures, leading to a false picture of
the progress and impact.
Non-Sampling Errors During Data Collection
sources of non-sampling error can be avoided or minimised. Table
6-7 lists some actions you can take to reduce the most common types
6-7. Common errors during data collection and how to reduce them
to Avoid Them
Make sure everyone understands the purpose of each method.
Make sure everyone knows exactly what data she/he is collecting
– clarify units, whom to speak with or where to go for data, and
the frequency of collection.
Practise interviewing and facilitation techniques.
Brainstorm about possible problems that might occur and agree on
various ways to avoid them or deal with them should they occur.
errors caused by poor documentation of data
Standardise formats for documentation.
Practise formats with the users and adapt the formats if necessary.
Computerise as soon as possible after data collection and check
the data entries.
Have enough material to record all responses and avoid losing data.
Pre-test questions and methods.
Present methods and questions (and especially their purpose) clearly
and confirm that people have understood.
Use local terms.
the Data Once You Have Them
must, from time to time, be verified. Only by checking whether your data
make sense and are valid can you feel safe that you are analysing progress
and process based on correct inputs. You do not have to check data all
the time. Keep your data verification process efficient by undertaking
spot checks at key moments:
all goes too smoothly with data collection, then probe to see if there
really are no problems lurking underneath the surface. Problems are inevitable
and their absence may signal that problems are suppressed. Keep an eye
out for signs of problematic data and investigate where problems might
accurate data. When the data collected match targets too perfectly
the data are probably problematic. In one IFAD-supported project in
Asia, large variations emerged in reporting per county. Most counties
consider the targets written in the appraisal report as compulsory
and strive to achieve them. They only report when achievements are
close to 100% of the targets. For instance, in two counties, the 1996
performance records a 100% achievement for practically all activities.
In another project, a review in 2001 of the data on physical progress
showed that targets and actual figures of implementation were exactly
the same, every month, for every parameter. These are clear cases
of unreliable data.
large changes in data
. In northeast Brazil, an NGO was monitoring
the adoption rates of contour ploughing and noticed a huge increase
in adoption rates. The NGO knew it had not undertaken much training
with farmers on contour ploughing so doubted the data. Focused research
was undertaken in several communities to see if the data were accurate.
It turned out that the data were, in fact, accurate but that adoption
to contour planting had been triggered by a surge in animal traction.
Animals cannot plough up and down steep slopes so contour ploughing
had become the side effect of increased use of animal traction. 9
in the data. When certain information has many non-respondents,
this may point to a respondent error or an error in the choice of
method for that information.
for Verifying Data
project needs to find its own way to incorporate verification into its
data-collection process. In Yemen, the RADP project deals with data verification
when management senses a problem with the data collected by component
departments and sent via the M&E unit. Management forms a committee
from the department concerned and the M&E unit to verify the information.
The department concerned may also make a field visit and submit a report
directly to the project director and copies to the M&E unit.
projects outsource data verification. In the ADIP project in Bangladesh,
the reliability and validity of data are crosschecked using additional
data-collection exercises. This includes, for example, the evaluation
of demonstration plot performance and research activities by consultants.
The responsible governmental department verifies M&E data, but project
management decides when such verification will happen and who should carry
it out. In the APPTDP project (India), the primary data are collected
through village liaison workers. Data are then verified by an appointed
agricultural/development consultant. Only then are the verified data passed
to the central monitoring unit for analysis.
check data yourself, triangulation is an important principle. This means
collecting the same type of information but from different sources and
using different methods. This can be as simple as, for example, asking
the same questions with different focus groups or comparing the outputs
of a map and a transect of the same area.
quantitative data is often more straightforward, as more agreed standards
exist. For example, many types of biophysical measurements indicate how
to calculate whether the data are representative. Verifying qualitative
data is more difficult, as there are no clear rules. You can use techniques
like "key judges" to verify the interpretation of information
(see Box 6-10).
6-10. Using different methods and "key judges" to verify
qualitative information in the Philippines10
the Philippines, the NGO, Education for Life Foundation (ELF), evaluated
its leadership-training programme. Various methods were used to
gather data, including focus groups (Method 12, Annex D), story-telling,
direct observation (Method 6, Annex D), psychological assessments,
surveys (Method 8, Annex D) and semi-structured interviews (Method
9, Annex D). As the information was mostly qualitative and open-ended,
the field researchers developed the idea of "key judges"
to cluster the information for analysis. They clustered and labelled
data according to topics they had selected earlier. Consensus was
needed by at least three people before labelling the data. The process
of data analysis allowed the researchers to share their different
interpretations of the answers and so it triangulated findings.
As a final check, they presented the draft findings to the communities
where data had been collected and they asked for feedback and suggestions.
knowing how to conduct interviews and facilitate discussions, fieldworkers
need to know how to record responses. Data can be recorded in many ways,
depending in large part on the data collection method. Some methods require
the filling in of forms or tables, others require using a tape recorder,
video recorder or camera, writing answers on cards or flip-charts, or
taking detailed notes.
each bit of information, define how it will be recorded. Practise with
the people doing the recording before setting out to collect data.
data-recording method you choose, make sure you are consistent in how
you record or it will be difficult to compare and analyse the data. Also
consider the information storage implications (see 6.4.4
for more details). Where and how will data be stored so that they are
safe and accessible? This will affect how data are recorded. Box
6-11 describes one example of the daily recording of information that
can then be fed into reports on the progress of the project.
6-11. Zimbabwean farmers record their day-to-day observations
one IFAD-supported project in Zimbabwe, farmers are asked to keep
daily records as part of the M&E system. The information they
record includes: production trends, gross margin budgets, cropping
programme (rotation), marketing trends (consumer consumption and
price comparisons), water usage per crop/plot, fertiliser use, pest
spraying programmes, scouting of pests and diseases, harvesting
outputs, labour costs and rainfall records. These records are then
compiled by the extension agent and submitted to the district agricultural
office for analysis. This provides monitoring information on the
scheme’s progress and is used to feed quarterly reviews and annual
work plans at the irrigation-scheme level. With systems like this,
it is paramount that farmers be supported to keep records accurately
and that data be verified regularly. Farmers will only be able to
sustain such high levels of information recording if it is meaningful
for them as well.
good form helps the recorder enter data consistently. It should clearly
represent the selected M&E indicators (as words, as diagrams or symbols,
or reformulated as a question) and give sufficient space for the collector
to fill in the information. Data forms should include space at least for:
location, time and duration of interview or discussion;
- name of
- name of
being discussed and methods used;
- key findings
– either in a predefined format (see Box 6-12)
or in terms of key words and descriptions if the data gathering method
is open ended.
6-12. Different options for predefined answer formats11
when the answer requires ticking one or more options from a list
(e.g., "Which health services do you use?").
questions: when the answer is "yes" or "no",
"agree" or "disagree".
questions: when there are several possible answers and you want
the respondent to consider all the possible answers before replying.
when you are asking people to give or rank their opinion. Ordered
scales are where people mark the statement with which they agree
and leave the others. An agreement scale requires respondents
to show the extent to which they agree with a statement, for example,
from "strongly agree" to "strongly disagree".
form the basis of discussion (see D.4,
and D.7), extra care
is needed to make additional notes since the diagram itself will never
capture all the important opinions and conclusions.
designs the recording form is critical, particularly if monitoring is
to be carried out by local groups. For example, self-help groups in a
credit project in India have developed a coding system to ensure that
all their members can participate in their regular self-evaluation process.
Since many members are illiterate, the questions are symbolised with pictures
and the three levels of evaluation are colour-coded. This is an example
of an agreement scale (see Box 6-12).
6.4 Collating, Analysing and
data have been collected, they need to be organised into a manageable
form ready to be analysed efficiently. This involves transcribing data
into a systematic format, entering the information obtained from each
respondent or group and organising it into one overall format, for instance,
into a computer database.
How to Collate Information
of information is needed when:
are scaling up your information from a smaller unit of analysis to
a larger one, for example, compiling all individual interviews to
develop an overview of a micro-credit group or pulling together all
village-level information into a district-level analysis;
has been collected from different sources with different methods,
to lay the basis for making comparisons and finding patterns during
of information requires an appropriate format. With some methods, this
is a very straightforward process. It can simply involve filling in a
statistical programme on the computer with numbers that represent measurements
or it can entail entering numbers that are pre-identified codes representing
specific ideas, following the form, questionnaire or notes used in the
data collection process. With statistical data, compilation ensures that
the many data are reduced to clearly labelled tables. These tables should
integrate the findings according to your performance question. For example,
it should show location-specific trends if you are trying to understand
how impact varies per community or district.
collation of qualitative data requires special care and analytical skills
(see Box 6-13). Box 6-14 describes
the basic steps to order open-ended responses. Section
6.4.2 discusses this in more detail, since data collation and data
analysis with qualitative data are overlapping processes.
6-13. Gaining confidence with qualitative reporting
from an IFAD-supported project in Indonesia are comfortable with
monitoring progress on physical, quantifiable indicators. They are
also confident that NGO partners working as implementing agencies
can use qualitative methods well enough for monitoring purposes.
However, they are unsure about how to report on information from
qualitative data – and on how to integrate it with physical progress
monitoring. This is understandable. It is often easier to fill pre-determined
forms requiring pre-determined information. This can be supplemented
by getting M&E and field staff to make regular descriptive reports
on their impressions from field visits. Initially, staff can write
short impressions of one or two pages. Once practised, staff can
focus their narrative reports on special aspects, such as poverty
alleviation, food security or gender.
6-14. How to synthesise and collate open-response information
a short summary of what each person says, including his/her main
over the responses. Once you are about a quarter of the way through,
note the points most frequently mentioned. Then read all the responses
and record how many have responded to each of these main points.
Alternatively, divide the responses into those for or against
a certain issue, or divide them to show various degrees of enthusiasm
about an issue.
any important quotes to emphasise certain points.
other people to look through the responses to prevent your own
biases taking over the way you interpret responses.
each respondent. Then, following point 2, number each main point
so that you can code the responses (who has noted a main point)
and analyse the information numerically, if needed.
Why Analyse M&E Information
M&E findings requires looking closely at the information (ideas, facts,
impressions), clarifying and structuring it, understanding connections
and identifying core elements, in order to arrive at conclusions that
can lead to action. Analysing M&E findings has several functions:
- to refine
understanding – by discussing initial information with project stakeholders,
more refined insights can emerge;
- to limit
biases – ensuring a thorough discussion about information means that
this is crosschecked and people can point out when they feel an issue
has been represented incorrectly;
- to build
a clear picture of a situation/event/process and reach consensus – by
discussing data, contradictions and gaps can be identified and can be
understood or filled;
- in participatory
M&E, joint analysis can strengthen ownership of the conclusions
and motivate people to invest more in making changes happen.
of M&E information and critical reflection are closely related, so
please refer to Section 8 for
many ideas on how to encourage reflective meetings and analytical reporting.
who needs to be involved in analysis. The question of who is making
sense of the data is central to participatory analysis. Often, work that
may initially have been very participatory can shift towards analysis
only by project staff. Sometimes this is necessary, as some aspects of
analysis and synthesis can be excessively tedious or time-consuming for
primary stakeholders. Shared analysis can make all the difference between
a superficial descriptive report or simplistic feedback session and analysis
based on shared understanding that motivates people to action, whether
they are villagers, policy makers or technical staff.
how you will undertake analysis. Choosing a method for analysis depends
on various factors, including whether it will be a participatory process,
the tool you use to collate and analyse the data (e.g., a computer), and
the type of information that is being collected. For instance, if it is
qualitative information, analysis will involve looking for patterns in
descriptions and explanations of patterns (see 6.4.3).
For quantitative data, the analysis will follow statistical procedures
and show trends in terms of percentages or ratios. In both cases, analysis
will involve comparing planned results with actual ones to understand
the reasons for differences, to compare differences over geographic ranges
or between groups, or simply to monitor changes over time.
of the methods in Annex D
can be used for data analysis. For instance, if you should choose more
participatory processes, see D.3 on discussion methods for more ideas.
D.6 on analysing relationships and linkages and D.7 for ranking and prioritising
are also useful.
Analysing Quantitative and Qualitative Data12
analysis of quantitative data is often better known in projects than that
of qualitative data. Quantitative data analysis often - but not exclusively
- encompasses calculations, such as total and average numbers of activities
implemented or percentages as compared to plans or targets. More elaborate
statistical analysis may also be required, for example with cost-benefit
analysis (see Annex D). Discussing the specific procedures of statistical
analysis lies beyond the scope of this Guide, so the focus here is on
ways to deal with qualitative information.
analysis of qualitative information is very different and can be more
difficult than that of quantitative data for those who are not used to
dealing with opinions and non-standard answers. Through content analysis
of collected information, conclusions can be formulated for each of the
performance questions or indicators. The analysis process involves identifying
the categories of responses found in the raw data.
the data collectors in analysis. All M&E data collectors and facilitators
– whether they are project staff, implementing partner staff or primary
stakeholders – should participate in sessions to analyse qualitative data.
Because of the nature of qualitative data, it is critical that those who
were present when the data were gathered also participate in analysis.
Much happens in open-ended discussions that is observed by facilitators
and helps to explain the data.
and analyse qualitative data concurrently. Qualitative data collection
is intended to trigger an iterative learning process. This means that
information from one discussion or interview will indicate aspects of
the topic that you will need to pursue with other questions and methods.
So analysis of one set of interview data may indicate changes needed in
subsequent interviews or discussions. A second reason for immediate analysis
of information is that it is impossible to note everything that is said
in open-ended discussions. Additional information, such as about the group
dynamics and how they influenced what was said, will not be recorded but
are critical to interpreting information. So the sooner the analysis takes
place, the easier it is to remember aspects that were not noted.
analysis around each performance question and each category of interviewees.
For example, if M&E field staff conducted individual interviews with
two farmer leaders and with the village council (VC) in one day, then
the two sets of data (farmers and VCs) would be analysed separately. During
the analysis, the team may need to refer back to the performance questions
to clarify the objectives of the different discussions.
these five steps to analyse the data.
the interview questions to the group. This allows everyone to remember
the focus of the M&E work.
note-taker(s) read aloud the responses for each question. If there
is more than one set of notes, each set of notes should be read.
the responses and share other comments that may not have been written
down, to clarify exactly what the interviewees were saying.
the responses and summarise findings. Together, identify the categories
of responses in collected information and summarise the findings concisely.
The summary should indicate the trends in the information in terms
of whether the attitudes or ideas expressed were shared by all interviewees,
the majority, half, a minority or only a few. Although you cannot
quantify the different types of answers, do report trends.
unclear or missing information. Determine whether there is missing
or unclear information that should be investigated in subsequent M&E
Storing M&E Information
information is critical for M&E, providing the basis for communication,
transparency, consensus building and continuity of consultative processes.
Stored information serves as the source of institutional memory turned
to by newcomers and when verification or comparison with the past is needed.
The quantity of information that all projects collect and share calls
for information systems to store data and make them accessible to others.
four questions when planning the storage of information (also see 7.5).
information needs to be stored?
about what information and how much you need to store. Information storage
is needed at two levels: to guide the project strategy and for tracking
operations. In principle, everything you decide to monitor and evaluate
will need to be stored in some way. Information about progress with implementation,
stakeholder reviews, annual project reviews, primary stakeholder databases,
changes in the context, causes, unexpected impacts, minutes of meetings…
the list quickly becomes overwhelming. Collecting excessive information
will also require you to store it (see Box 6-15).
Therefore, consider carefully what information needs to pass to whom for
decision making and for reporting. Section
5 details how to choose what to monitor and evaluate.
6-15. What you store is just as important as how you store it
the surface, the information management system in an IFAD-supported
smallholder cattle development project in Asia looked good at the
end of two phases. It was filled with extensive data from the project
and had been computerised and updated. However, several flaws in
the system impeded project impact assessment. For example, despite
extensive staff training, it proved too formidable a task to enter
more than ten years of data for all project activities. The data
overemphasised physical achievements and credit repayment, with
no monitoring of farmers’ perceptions of how they had benefited.
Socio-economic indicators were lacking in many ways. There were
technical flaws in the selection of respondents and the size of
questionnaires, etc. Historical records were not kept on loan repayments.
Furthermore, most of the survey data were not analysed. This did
not allow for time-series data analysis and, therefore, impact could
not be measured.
needs access to the information and when?
the data are stored depends on who is to have access to the information
and how often. Information to guide the project strategy is critical for
managers (project staff and implementing partners), steering committees,
primary stakeholder representatives and funding agencies. Information
on operations is critical for fieldworkers, managers of project components
and primary stakeholders.
the skills of the users and the types of communication with which they
are comfortable (see Box 6-16).
store material where it will be used. This is particularly important with
the raw data on paper, such as diagrams. Do not assume that all diagrams
need to be copied, distributed and stored at all levels. Only keep them
where they are used. This usually means leaving the originals with the
stakeholders who produced them.
6-16. The advantages of decentralised computer-based data storage
data processing system can form the basis around which to decentralise
and encourage ownership through participatory collection, recording,
analysis and reporting. This is the case for the automated monitoring
system of the Cuchumatanes project in Guatemala. There, the M&E
unit only needed to review the quality of the data gathered and
manage the information at the central level. The field implementers
were trained to use the computerised storage system and every region
had access to its own information. Managers of each implementing
partner were responsible for feeding the system, producing the reports
and sending them to a central M&E unit. The automated system
was eventually transferred to an implementing partner after training,
and the project M&E staff maintained access through the electronic
network. This set-up allowed for each organisation to know its status
in relation to its annual work plan and also to have timely information
for local decision making.
type of information needs to be stored – hard copies or data that
can be computerised and accessed centrally?
more people who need to use information, the better it is to computerise
it. However, not all data gathered at a local level will be entered into
a computer. This can be due to local implementing partners and primary
stakeholders not having access to computers or electronic networks or
lacking the necessary skills, or because the information is diagram-based.
Diagrams can be (photo-) copied and distributed to those who will need
access in that form, for example, local groups and community-based facilitators.
Generally, however, you will only need short reports that summarise the
findings from the discussions that occurred as the diagrams were generated
and from the diagrams themselves.
assess what information you need to keep and what can be discarded.
data-storage system will soon get congested and overflow if it is not
updated regularly. This is as true for archives of hard copies as it is
for computerised data. Computerised data are more easily archived in unobtrusive
yet accessible ways. Simply make backups and store them in a safe place
away from the hard disk.
hard copies, making decisions about what to discard is more critical.
Make sure that you keep all material you are legally required to store,
such as tax and audit-related financial records, for the required time
period. This will vary per country. Also make sure that you keep copies
of all material you need for making comparisons of change over time. This
includes baseline data, summaries of progress with implementation and
interim impact information.
6.5 Communicating M&E Findings
for Action and Accountability
Why Communicate M&E Findings
findings have many potential audiences. When reporting on progress with
the AWPB, you will direct yourself to funding agencies, steering committees,
cooperating institutions and implementing partners. Primary stakeholders
have a right to knowing overall how the project is progressing and they
deserve the opportunity to react to initial findings. Funding agencies
and managers need information on impact, while all implementing partners
need to understand problems in order to find solutions.
sets of M&E findings will need to be communicated. First, it is good
practice to discuss draft M&E findings with implementing partners
and primary stakeholders in order to get feedback on accuracy, reach joint
conclusions and agree on next steps. Once the M&E findings are agreed
upon, these can be communicated to funding agencies, cooperating institutions,
government departments and other projects. This second set of final findings
will fulfil accountability needs but can also serve for advocacy purposes.
Planning How to Communicate M&E Findings
agreement with project stakeholders on who needs to receive what kinds
of M&E information. Table 6-8 shows the information
needs of different audiences for a World Food Programme project in China.
It outlines what data and insights the M&E system must produce and
for whom. Note that it focuses on communicating for accountability and
not on communication for action and decision-making purposes.
undertaking an audience analysis for your project, remember to:
6-8. Audiences for information on a WFP project in China
13 (high/medium/low priority)
of Project Information
Communication Into Your M&E System
not hope or expect that someone else in the project will communicate
M&E findings. Plan for it from the onset. In Ghana, a workshop was
organised with different M&E actors in order to develop a flow chart
for the routing of M&E information from the grassroots level up
to project management. The flow chart identified and offered solutions
to communication bottlenecks in the M&E system, plus it identified
who was responsible for the different information flows and established
necessary frequency and deadlines for report submission. By discussing
and planning for these communication issues, the M&E system was
more likely to operate smoothly (see Box 6-17).
6-17. Information flows in Zimbabwe’s SISP ensures feedback, action
on the indicators from all the irrigation schemes is fed into
annual plans. In turn, these scheme-level annual work plans and
budgets feed into the district-level planning process, and the
outputs of which are used to plan at the national level. Although
the provincial level is not involved in SISP monitoring, it will
receive information about activities per component in the form
of progress reports. Once the information has been synthesised
at the national level, the findings will be communicated back
to the districts and to the schemes, first in the form of changed
or consolidated priorities and work processes (feedback and action)
and then in the form of newsletters at the scheme level (feedback).
only do members of individual irrigation schemes learn about their
own progress through M&E, but they are also able to view the
data related to other schemes and so can compare their own performance.
In addition, receiving information on the institutional performance
of SISP is critical. These types of feedback ensure that the stakeholders
remain accountable for their actions.
in Good Communication
good communication strategy can generate more support and interest in
your project – it is worth the investment. Box 6-18
lists some elements that made the communication strategy of the Maharashtra
Rural Credit Project in India a success. They include professionally
prepared presentations of progress and constraints, which were used
with positive results at high-level meetings. Investment is not only
in terms of producing effective outputs but also in project-based capacities
(see Box 6-19).
6-18. Linked and complementary M&E documentation
in the Maharashtra Rural Credit Project in India included these
complementary ways of reporting on self-help groups (SHGs).
pictorial self-monitoring system for self-help groups helped
in monthly and annual monitoring. The system was composed of
a three-category ranking system to be used for 16 indicators
– ranging from the quality of meeting preparations, to repayment
records, to collective decision-making.
district reports captured process issues in the formation of
village development committees and the SHGs.
National Bank for Agriculture and Rural Development produced
a newsletter that provided information on the progress of the
SHGs at the district and overall-project levels.
analysis of the project’s progress was presented at high-level
meetings and included clear and understandable graphic representation
of the trends.
6-19. Production manager is recommended to support appropriate
communication strategy in Zambia
communication strategy recommended a production manager for the
Zambian District Development Project for:
to be translated into languages and formats well-suited for
the target audience(s) in a timely manner;
labour-intensive and time-consuming process to move communication
ideas (text, images, concepts, etc.) into products, field testing
and quick dissemination;
transparent communication flows.
strategy necessitates a production manager who:
experience in developing participatory communication materials
the strengths and weaknesses of these materials and methods;
extensive experience with a wide variety of vendors, from printers
to graphic artists to photography studios;
work closely with the technical person in the project support
unit and coordinate the process of moving the materials from
their raw stage to the final product.
Workshops to Seek Feedback and Plan Action
key communication task is to ensure that your findings are correct.
For this, you need to organise feedback sessions with those stakeholders
who can verify findings. This is also a good moment to analyse implications
and agree on actions. You can include this in your plan for the sequence
of methods (see Box 6-20). Also refer to Section
8, which offers many ideas on how to ensure that an M&E event
is communicated and reflected upon, and thus is more likely to lead
6-20. PRA sequence with key feedback sessions14
Used and Their Purpose
of secondary data
review (literature review, in-country sources and grey literature)
informant interviews and focus groups (structured/semi-structured)
– at the national, regional and local (government, primary and
other stakeholders) levels
walks (local level)
collation and analysis
trends, maps, pie diagrams and such with field staff (extension
agents, M&E unit, etc.) and primary and other stakeholders
survey ("last step")
biophysical measurements during a transect, group discussions,
etc. to gather information to cover unanswered questions, fill
gaps in the data and substantiate controversial findings
the national (project management, relevant ministries, donors
(IFAD, etc.) and local (primary stakeholders) levels
seminars (to check validity and pertinence of results pertaining
to project goals, activities, ongoing efforts)
Considerations When Presenting Information for Feedback and Action
clarity of message for specific audiences. The interests and concerns
of different audiences vary and will require adapted reports, both
in terms of content and language. Reports should communicate different
levels of detail according to the audience being addressed. For example,
strategic and implementation levels of management require different
focuses. At the strategic level, you need to provide a general review
of the project’s progress and problems. At the implementation level,
more detail is required to help facilitate and coordinate day-to-day
project management tasks.
on the frequency for communicating information. This will often
fit the timing of decision-making meetings. If you are holding a meeting
in order to seek immediate feedback, choose a time when people will
be able to come.
timeliness. Be sure to present information while there is still
momentum, in order to benefit from the feedback. However, if setbacks
should take place, be sure to let the audience know and be clear about
the delay involved. This issue is not only important for getting feedback,
but also for maintaining project credibility.
. Box 6-21
shows the importance of
thinking about various conduits of information to be sure that how
and where you share your findings will be able to reach people, providing
them the opportunity to give feedback.
6-21. Remember to tap into informal conduits of information
consultation processes have been integrated into the District
Development project in Zambia as an essential part of the project’s
M&E system. Those involved in the consultations are considered
to be important conduits into other formal and informal village
information dissemination processes. Issues of interest in rural
communities can pass quite quickly through informal channels such
as markets, social events (church services, etc.), and weddings
or funerals. Focused communication campaigns do not penetrate
these informal channels easily. Informal venues provide an excellent
opportunity for social discussion as people feel more comfortable
in these settings to ask questions and talk, forming individual
and group opinions.
effective use of graphic information to facilitate analysis. Visually
presented information is often easier to understand. The better and
more quickly your information is understood, the more likely you will
get direct, useful feedback. There are many ways to present your information
pictorially: through the use of graphs, diagrams, maps, pictures,
photographs or videos. Some of these presentation forms will arise
naturally as a result of your choice of data-gathering method. For
instance, by showing the results of a series of mapping exercises
or photographs, people can see at a glance what has been measured,
how and how it has changed. Other visual portrayals, such as graphs
or pie charts, need to be created from the information obtained through
statistical data analysis.
focused on your task. A feedback session can strand in a general
talking event with no clear outcomes. Plan the event carefully around
the anticipated outputs – e.g. clarifications, additional insights,
conclusions, action steps, etc. Don’t rely on improvisation as your
main facilitation strategy. It is always necessary but too much can
lead to confusion. Avoid imposing ideas by thinking how people are
most likely to share their thoughts on the M&E data. Be sincere
in the reporting – include the new insights, otherwise participation
will become a farce.
Different Media to Communicate Findings
reports vary from formal progress reports, to special studies, to informal
briefs in the form of memorandums highlighting a current issue. Most
IFAD-supported projects produce annual work plans and budgets, quarterly
and mid-year progress reports (see Box 6-22), a
mid-term review and a completion report. Some produce annual reports
and many have newsletters (see Box 6-23). A small
booklet of stories and photographs was used to report on the impact
of Ireland Aid’s Water and Environmental Sanitation Programme in Western
Uganda. As mentioned in the introduction, "It is important to recognise
and record the impact of development projects on individuals’ lives,
as felt by the people themselves. By listening to their voices, hearing
their stories and learning from them, we begin to understand the impacts
of development assistance on daily life from people’s own perspective,
and put a ‘human face’ on a programme’s impact through the use of photographs,
stories and oral histories."
6-22. Using a logframe to guide reports in Colombia
PADEMER, reports from the implementing partners have been streamlined
to focus on the logical framework structure. This allows a clearer
overview of the effects and impacts that were hoped for (in accordance
with the formulated indicators) and of the activities with which
they would be achieved. Partners were trained in using this format.
Formats were also made to present technical and financial reports
per trimester. They are simple reports that allow a clear view
of what each project is doing. Subsequent payments depend on the
presentation of good reports. Reports are expected 1) to be brief
and objective and take down only information that is basic and
indispensable, 2) to present the current state of actions based
on the programming and data of the approved logical framework,
and 3) to be submitted in printed form and on diskette, by electronic
mail, and using predefined structures, such as the one below.
and Finish Date)
of the activity:
was done and how:
of the activity:
encountered and solution(s):
6-23. Newsletter communicates M&E findings to farmers in Zimbabwe
farmer involved in an IFAD-supported project in Zimbabwe collects
various records regarding her/his agricultural production, including
production and marketing trends, water and other input usage,
etc. This information is usually compiled by the extension worker
and is submitted to the district agricultural extension office
for production analysis. In addition, a regular newsletter is
produced containing information on trends, ideas and project progress
as well as farmer interviews. This focuses on the latest project
developments, how the information that farmers have produced is
used, and decisions made at the national level.
information bulletin produced for the IFAD-supported Ngöbe-Buglé
project in Panama has been developed into an attractive means
of communicating information on project progress. With a few simple
graphic designs to frame the pages, photographs, a map of the
project area and a clear layout, it has become an effective communication
tool. The bulletin contains evaluation information on project
activities – with shorter reports on meetings and events – and
other important occurrences in the project.
M&E findings can be communicated more effectively verbally than
by other means. Much decision making is based on information obtained
through personal contacts and oral presentations. To speak directly
to a target audience provides a quicker and more flexible way to convey
your message. You can modify your presentation according to the feedback
you receive. When conducted well, face-to-face contact can lead to greater
understanding and more frank discussions on your findings. Bear in mind
that some information may be better conveyed in individual rather than
in group meetings.
can also be effective. In one project in Peru, 20 farmers’ radios provide
daily information on current activities, project-related decisions,
resources to be transferred to the communities, meetings, visits, and
interviews with farmers and extension agents. The radio plays an important
M&E function by disseminating information and decisions and motivating
displays, such as graphs or charts showing trends or maps, help illustrate
and supplement data in reports or oral presentations. You can also choose
to photograph or shoot video images on changes (see Method 20, Annex
D). Photographs can bring a project or community alive in a way
not possible through words and diagrams. Dramatic presentations, whether
on video or live, can be another good way to communicate insights with
greater impact than on paper.
more creative, however, can mean more time and money to develop the
idea and train (or hire) people in necessary skills. This needs to be
considered when looking at alternatives.
8 provides valuable additional information on critical reflection
that is fundamental when communicating M&E data. Sections
3, 4, 5,
and 8 include additional material on reporting of M&E-related information.
J. 2000. Participatory Program Evaluation Manual: Involving Program
Stakeholders in the Evaluation Process. Calverton, Maryland: Child Survival
Technical Support Project (with Catholic Relief Services). Download
in English, French and Spanish
E. 2001. Facilitation Made Easy: Practical Tips to Improve Facilitation
in Workshops. London: Kogan Page. Order
D. J. and Kumar, K. 1988. The Collection, Analysis and Use of Monitoring
and Evaluation Data. Washington, D.C.: World Bank. Order
D. J. and Kumar, K. 1990. Project Monitoring and Evaluation in Agriculture.
Washington, D.C.: World Bank. Order
M. T. 1986. Partners in Evaluation: Evaluating Development and Community
Programmes with Partners. London: Macmillan. Order
I. 1998. "Participatory Monitoring and Impact Assessment of Sustainable
Agriculture Initiatives: an Introduction to the Key Elements".
SARL Discussion Paper No. 1. London: IIED. Order
or contact firstname.lastname@example.org
ANGOC and IIRR. 2001. Enhancing Ownership and Sustainability: A Resource
Book on Participation. International Fund for Agricultural Development,
Asian NGO Coalition for Agrarian Reform and Rural Development and International
Institute of Rural Reconstruction. Contact: email@example.com.
R. and Salafsky, N. 1998. Measures of Success: Designing, Managing,
and Monitoring Conservation and Development Projects. Washington, DC:
Island Press. Order
UNDP-World Bank Water and Sanitation Program - South Asia. 1999. Sustainability
Monitoring: the VIP Way. A Ground-level Exercise. UNDP-World Bank.
Gosling, L. with Edwards, M. 1995. "Toolkits: A Practical Guide to
Assessment, Monitoring, Review and Evaluation". Save the Children
Development Manual 5. London: Save the Children.
Johnson, D. with M. W. Kanyinji, R. Lupenga, G.L.E. Nkhata, B. K. Siantumbu
and S.M.H. Sithole. 1998. Communication Strategy and Monitoring and Evaluation
Framework: Main Report. Zambia: District Development Project, Government
of the Republic of Zambia and the United Nations Capital Development Fund.
Westley, K. "Lessons Learned from Different CARE Country Offices
in Design, Monitoring and Evaluation". Mimeograph. London: CARE-UK.
IFAD, ANGOC and IIRR 2001, 240, see Further Reading.
Note that "appropriateness" is not defined in this example.
Herweg, K., K. Steiner and J. Slaats (eds). 1998. Sustainable Land Management
Guidelines for Impact Monitoring: Workbook and Toolkit. Page 65. Berne:
Centre for Development and Environment.
Abbot, J. and Guijt, I. 1998. "Changing Views on Change: Participatory
Approaches to Monitoring the Environment". SARL Discussion Paper
2. London: IIED. Download
Inspired by: Mikkelsen, B. 1995. Methods for Development Work and Research:
A Guide for Practitioners. New Delhi, London, and Thousand Oaks: Sage
Publications. and Rugh, J. 1986. Self Evaluation - Ideas for Participatory
Evaluation of Rural Community Development Projects. Oklahoma City: World
9/ Sidersky, P. and Guijt, I. 2000. "Experimenting
with Participatory Monitoring in Northeast Brazil: The case of AS-PTA's
Projeto Paraíba". In: M. Estrella et al. (eds). 2000. Learning
from Change: Issues and Experiences in Participatory Monitoring and Evaluation.
London: Intermediate Technology Publications.
Abes, R.V. 2000. "Strengthening citizen participation and democratisation
in the Philippines: ELF's impact evaluation." In: M. Estrella et
al. (eds). 2000. Learning From Change: Issues and Experiences in Participatory
Monitoring and Evaluation. London: Intermediate Technology Publications.
Feuerstein, M. T. 1986, 95-102, see Further Reading.
Based on Aubel, J. 2000, see Further Reading.
Hereward, M. and Siping, W. 2000. "Bringing results to the people".
Presentation at M&E Workshop Wuhan, 9 - 12 October, 2000. UNICEF Office
Modified from FAO, "Notes on Rapid Appraisal Methodologies for Assessing
PDF Version (259 KB)