Annex C - Annotated Example of an M&E Matrix

This Annex provides an example of the M&E matrix (see C.3, Table C-2) that was introduced in Section 2 and explained in Section 5.

To establish a good M&E system, you will need considerably more detail about how to gather and use information than can be summarised in the logframe matrix. Using the M&E matrix is one way of identifying and documenting this additional information.

It is important to recognise that an M&E matrix is only part of an overall M&E plan, as it only considers what is needed to monitor and evaluate the objective hierarchy. It provides detailed information about how the goal or particular components, outputs and activities will be monitored and evaluated. An M&E plan will include other events that make it possible to understand the project context, reflect and learn lessons.

Completing the M&E matrix requires detailed knowledge of the project and the project context. As the example in Table C-2 is hypothetical, it cannot include the level of specific details that would exist in a real project situation. Consequently, it is more general than a real matrix would be. The example aims to provide an overview of important aspects of the matrix, rather than present a fully detailed matrix. The example is based on the logframe matrix developed in Annex B.

C.1 Considerations When Developing an M&E Matrix

As you read through the M&E matrix example, there are several points to keep in mind. These might be of use to you in understanding the example matrix but also when developing your own M&E matrix.

From data to lessons, understanding and decisions. When developing the matrix, it is important to keep in mind how you plan to move from data collection to explaining success or failure, creating understanding about particular issues with stakeholders and, finally, making decisions. In the example, the proposed annual workshop on food security will enable you to reach a decision based on focused reflections. This workshop will also allow you to identify lessons learned that can inform next year�s work.

The matrix at different levels in the objective hierarchy. In the example, you will notice that at lower levels in the objective hierarchy it becomes much easier to be very specific. For example, monitoring the length of main and secondary irrigation canals dug is a more straightforward task than assessing overall contribution to people�s livelihoods. The further up the objective hierarchy you go, the more analysis and synthesis of different types and sources of information are required. Note that in the formal logframe matrix (see Section 3), indicators and monitoring mechanisms are not entered into the logframe for activities. Instead, resource inputs and costs are used. However, in practice you will still need to monitor the activity level so you still need to plan for this. The M&E matrix can be used to plan what is needed for the activity level as well as for the goal level.

Triangulation and validation. When deciding what data-gathering and analysis methods to use, think about how you can ensure that data is reliable. Triangulation means getting information about the same topic in a number of different ways. Validation is what you achieve by then cross checking the information. For example, from the participatory impact monitoring (PIM) with women�s groups you might get feedback that a particular new enterprise is very time-consuming in relation to the income earned. If you get the same feedback from household surveys and field observations of staff, you can be more confident of the information. On the other hand, if you are getting conflicting information from these different sources, then you will need to investigate further to understand why there are differing opinions.

You can also use the field records of agricultural extension staff or the government as sources of valuable information for project monitoring. However, positive aspects may be exaggerated and problems overlooked � an inevitable human tendency. It is thus important to have methods in place for checking and validating information.

Existing information and data-gathering systems. Most projects will have access to the already existing data-gathering and statistical systems of the government or another agency. It is critical to see how these can be used. It may be that for a small investment, existing systems can be improved or modified to meet the project�s monitoring requirements.

Technology use. Consider carefully where and to what extent information technology can be effectively used. In the example, the increased areas of new crops could potentially be monitored using remote sensing and geographic information systems.

Specific data vs. the big picture. Specific indicators provide fragmented bits of information. To thoroughly understand the project, its successes, failures and lessons, you need to build up the overall story. This means integrating and analysing different pieces of information. The "analysis, reporting, feedback and change" column of the matrix provides a start for thinking about how this can be done.

Primary data or secondary observations. Information about changes in household capacity to meet education, health and housing needs could be gathered in two ways. A detailed household survey could be conducted (primary data) or community representatives and other key informants could be asked about their observations (secondary observation). Generally, primary data is more reliable but also much more time-consuming and costly to collect. An important skill in completing the M&E matrix is being able to balance the use of methods with the required level of information accuracy and the available resources. Think carefully about whether very detailed information is needed or whether an understanding of the general trend is adequate. If all you need is a general picture, then it might be possible to use qualitative methods in a cost-effective manner.

Working with specialists. To monitor some parts of a project may well require specialist advice and input. In the example, it is proposed that an economist be used to conduct a survey of the local economy. Likewise, monitoring the yields of different agricultural crops is a specialised task for an agronomist. There are now many people with strong experience in monitoring microfinance schemes. Wherever possible, try to draw on such expertise. The role of the M&E specialist is to link this expertise to the project�s M&E plan and assess how detailed the M&E needs to be for different parts of the project, given resource and time constraints.

Aggregation of field data. It will often be necessary to aggregate data from different field locations. In the example, agricultural extension staff are involved in recording changes in cropping patterns at district and local levels. If these aggregation processes are to generate reliable information, you will need to develop recording forms and aggregating systems. You can detail this in column five of the M&E matrix.

C.2 Core M&E Activities in the Example

Project M&E will require you to use different methods to meet a wide range of information needs. When developing a project M&E matrix, you may feel as if you are drowning in the detail of methods for each information need. However, a second look will reveal clusters of M&E activities. While the M&E matrix requires you to specify information needs and methods in detail, in practice they converge. This means that one M&E activity can be used to meet multiple information needs.

In the M&E matrix example (see C.3, Table C-2), you will find seven major recurring M&E activities (see the list below). For example, a household survey will provide information for a range of performance questions and indicators. Information from such a survey may be combined with information from other sources, such as participatory impact monitoring, to inform the annual project review.

The M&E matrix example focuses around seven major activities that form the basis of this project�s M&E.

Participatory rural appraisal (PRA). Three PRAs will be undertaken: the first, during the mobilisation phase; the second, one year prior to the mid-term review; and the third, two years after project completion. The PRA work will provide largely qualitative information about the needs, issues and perspectives of different stakeholder groups. The PRAs will be important in gathering information about unintended positive and negative impacts from the project.
Household and farm surveys. Sample household and farm surveys will be conducted to gather necessary baseline information, support the mid-term review and conduct an evaluation two years after project completion. In addition, more limited surveys will be conducted on a yearly basis.
Participatory impact monitoring (PIM). Participatory M&E systems will be established with key stakeholder groups involved in project implementation, such as farmers� groups and women�s groups. This will involve the stakeholders in setting their own performance questions and questions, developing monitoring systems and participating in training to support implementation.
Stakeholder discussion groups. A series of stakeholder discussion groups will be established around key project components and outputs. These groups will help to analyse and review information, identify lessons learned and make recommendations about necessary changes in the project design.
Government statistics. Government normally collects a range of population, economic and agricultural statistics relevant to project M&E. Improvements in some aspects of this data gathering will be supported by the project and the reliability of the data will be assessed.
Field observations. All project and implementing partner staff will continually undertake systematic field observation. This will require producing and using key questions relevant to their area of work, training and information gathering and synthesis procedures.
Special studies. A range of special studies will be conducted, for example, on changes in the structure of the local economy. Some of these studies will involve external expertise. Where necessary, special studies will be undertaken to provide further information on important issues or opportunities that emerge from the regular monitoring work.

Each of these M&E activities needs to be well thought out and planned. When planning the complementary use of the activities, it is particularly important that you determine which ones can help provide information for which specific performance questions and indicators in the matrix.

C.3 The M&E Matrix Example

Table C-2 shows a partial example of an M&E matrix with annotations. The matrix includes several different levels from the objective hierarchy but has not been completed for the entire project. As mentioned above, it is a hypothetical example and so the details are not as precise as they should be in a real example. The purpose of the example is to give a general idea of the types of issues that need to be considered when developing an M&E matrix for a real project.

Table C-1 provides annotations for the M&E matrix example. The numbers in the table correspond to numbers in the matrix.

Table C-1. Explanation of the issues highlighted in the M&E matrix example

Number

Issue

Explanation

1

How to present information

Think about how information needs to be presented for it to be meaningful. Information should usually be presented in comparison to a target, a prior state or the original state. For example, knowing only the number of households who are more food secure is not as informative as knowing the percentage and how the percentage has changed over time.

2

Responsibilities

In the example, responsibilities have not been included. However, for a real project it is important to identify who will undertake what aspect of the M&E work.

3

M&E activities that provide information for several performance questions and indicators

Some M&E activities, such as a household survey, PRA or PIM, can provide information for different evaluation questions and indicators. The M&E matrix can help identify the different types of information that need to be gathered from such activities. The details of how these activities will be conducted and the resource implications should be outlined separately to the matrix.

4

Combining and analysing information

In planning the M&E system, try to think as much as possible about how different information can be combined and analysed to report progress and also to explain success and failure.

5

Review groups

Don�t let good information sit on the shelf. The project can set up different review or working groups or hold annual workshops with key stakeholders to track progress and identify lessons learned for different aspects of the project. These groups or events can then feed their conclusions into the annual project review process. The information needs of these groups can help refine the overall monitoring and information-gathering strategy.

6

Reasons

Collecting information about why part of the project is succeeding or failing is just as important as monitoring what has been achieved. Collecting information about reasons generally requires gathering and analysing qualitative information.

7

Specialist studies

An economic study such as this is probably beyond the project team�s capacities and would require input from an economist.

8

Technical methods for monitoring

Some indicators will require specialised technical methods related to particular disciplines and specialisations, in this case agronomy. It is very important that proper technical expertise be used when developing such monitoring mechanisms.

9

Participatory impact monitoring

Monitoring yields at the field level directly could be very costly. It may be possible to obtain adequate information through discussions, in this case, with farmers� groups.

10

Field inspections to validate data

Validating data is critical and field inspections by project M&E or other staff is one way to do so.

11

Using other sources of expertise

In this case, a university has been used to undertake a specific monitoring activity.

12

Setting criteria

An indicator will often include a quality, like "operating effectively", that must be defined. In this case, it would be necessary to identify the criteria for effective operation, such as "regular meetings with 75% of members".

13

Monitoring by implementing partners

Much M&E can be undertaken by implementing partners. However, it is important that they be involved in designing the M&E system and be supported to carry out their responsibilities.

Table C-2. Example of a partial M&E matrix

Goal: Improved livelihoods for 35,000 poor families in the Rutunga Province through increased food security and enhanced income-generating activities
Performance Questions and Related Targets	Information Needs and Indicators	Baseline Information: Requirements and Status (If Known)	Data Gathering: Methods, Frequency, Responsib- ilities	Planning and Resources: Forms, Planning, Training, Data Management, Expertise, Responsib- ilities	Information Use: Analysis, Reporting, Feedback, Change Processes, Responsib- ilities
For whom has food security changed and in which ways? 75% of families with food security under average seasonal conditions	Changes over time in per cent of total households who are able to meet minimum nutritional requirements (disaggregated according to type of household, season and location)	Per cent of households with food security under average seasonal conditions at start of project - estimated at 40% from 2001 survey	Sample household nutrition surveys: baseline, mid-term, project completion, three years after completion PIM programme with sample villages and women�s groups (ongoing)	Nutrition survey to be included in household survey (See separate details on household survey methodology.) Nutritionist with M&E experience to provide specialist input (See separate details.)	Household survey information to be compared with feedback from PIM, PRAs and field observations Yearly workshop with key stakeholders on food security
	Reasons for changes (for those now and those still not/no longer meeting requirements)	Reasons for insecurity at start of project	PRAs: baseline, mid-term, project completion, three years after completion PIM: ongoing	See details on PRAs and PIM.	As above
How has the purchasing power of target households changed � in particular, for housing, education and health needs? 30% increase in household expenditure on housing, education and health	Changes in income, costs and expenditure patterns (disaggregated according to type of household and location)	Household expenditure patterns at start of project	Sample household surveys: baseline, mid-term, project completion, three years after completion PRAs: baseline, mid-term, project completion, three years after completion PIM: ongoing	See details on household surveys, PRAs and PIM.	Annual and mid-term review of project with key stakeholders about the project�s contribution to overall livelihood improvement in the context of other initiatives in the province
How have project interventions influenced the meeting of housing, education and health needs?	Changes expenditure patterns (disaggregated according to type of household and location) General observations	Status at beginning of project	PIM Reporting by NGO and agricultural extension staff PRAs	See details on PRAs and PIM.	Identification of necessary changes and options for better collaboration with other initiatives
How has the diversity and size of the local economy changed?	Changes in types and value of products and services being exchanged	Nature of local economy at start of project	Economic analysis of local economy to be undertaken by specialist/ economist: baseline, mid-term, project completion, three years after completion Use of government statistics	Development of methodology and analysis by specialist/ economist Strengthening of government statistical methods for the province	Discussion of economic changes and developments during annual project review Establishment of economic development task force
	Proportion of benefits of economic growth that stays local	Nature of local economy at start of project
How have interventions affected the workloads, roles and well-being of different household members (women, men, young, old)?	Changes in workloads, roles and well-being disaggregated by gender, generation and household type	N/A	Household surveys PRAs PIM	See details on household surveys, PRAs and PIM.	Regular discussion with women�s advisory groups
How equitably have different social and economic groups benefited from the project�s interventions?	Analysis of all above information in relation to household members and household type	Identification of particularly disadvantaged groups at start of project	Household surveys PRAs PIM	See details on household surveys, PRAs and PIM.	Discussion of changes and developments during annual project review

Component (purpose) 1: Agricultural production � agricultural production increased, and diversified in a sustainable way
Performance Questions and Related Targets	Information Needs and Indicators	Baseline Information: Requirements and Status (If Known)	Data Gathering: Methods, Frequency, Responsib-ilities	Planning and Resources: Forms, Planning, Training, Data Management, Expertise, Responsib-ilities	Information Use: Analysis, Reporting, Feedback, Change Processes, Responsib- ilities
How have the cropping patterns in the target area changed? Area of horticulture and vegetable production increased to 4,000 hectares Area of non-rice crops increased by at least 10% for small farmers	Changes in cropping patterns disaggregated according to location and farmer type	Land use and agricultural activity information at start of project (from department of agriculture)	District record-keeping by department of agriculture field staff � quarterly data and twice-yearly and yearly analysis and reporting Remote sensing	Improve department of agriculture�s record-keeping procedures, train staff and develop a new reporting format. Install a database and geographic information system.	Agricultural production task force will be established. It will meet every three months and review monitoring data. It will make a report for the annual project review meeting, covering progress, lessons learned and how any problems can be overcome.
How much have farmers increased their yields of specific crops 60% of farmers achieving 70% of target yields in years with average seasonal conditions	Changes in average yields per crop (disaggregated by location, year and crop types)	Yields at start of project	Sample field surveys at harvest PIM by farmers� groups	Identify sample sites and train staff in field measurement. Develop participatory yield-appraisal and recording system with farmers� groups.
What innovations/ improved farming practices have been developed or recommended? What level of adoption has occurred? What are the reasons for adoption or non-adoption?	Types of innovations/ practices that have been developed and recommended Level of adoption of different innovations Reasons for adoption or non-adoption	N/A	District record-keeping by department of agriculture field staff � quarterly data and twice-yearly and yearly analysis and reporting PIM by farmers� groups	Improve department of agriculture�s record-keeping procedures, train staff and develop a new reporting format.
How have the environmental impacts of agriculture changed? 70% of farmers adopt at least one environmentally sustainable practice Chemical load in Besha River reduced to target levels	Level of adoption of environmentally sustainable practices	Extent of environmentally sustainable practices at start of project	District record-keeping by department of agriculture field staff � quarterly data and twice-yearly and yearly analysis and reporting Field inspections by project staff	Improve department of agriculture�s record-keeping procedures, train staff and develop a new reporting format.
	Levels of indicator chemicals Silt load	Levels at start of project	Chemical analysis of water samples every month	Give monitoring contract to Ingsar University.	Results will be discussed at provincial environmental committee.

Component 3: Institutional development � Output 1.1 farmer support groups established and operating self-reliantly

Performance Questions and Related Targets

Information Needs and Indicators

Baseline Information: Requirements and Status (If Known)

Data Gathering: Methods, Frequency, Responsib-ilities

Planning and Resources: Forms, Planning, Training, Data Management, Expertise, Responsib-ilities

Information Use: Analysis, Reporting, Feedback, Change Processes,
Responsib-
ilities

How effectively are farmers� groups supporting farmers to adopt new crops and improved farming systems?

500 farmers� groups operating effectively

Number of farmers� groups

Number at start of project

Records from department of agriculture field staff and from officially registered groups receiving financial support from the project

Development of record-keeping forms

Analysis and discussion of success of farmers� groups within agricultural extension support group

Per cent of target farmers actively involved with a farmers� group

Per cent at beginning of project

Recording-keeping by farmers� groups and aggregation and synthesis by M&E unit

Development of record-keeping forms

Number of farmers� groups meeting criteria for a successful group

Number at start of project

Reporting by department of agriculture field staff

Qualitative survey of farmers� groups every two years

Record-keeping by supporting NGOs

Establishment of criteria for a successful group

Training staff to undertake a farmers� group survey

Development of record-keeping system for NGOs

Extent to which farmers� groups have influenced adoption of new practices

Historical role of farmers� groups

Interviews with key informants

Component 5: Infrastructure built and maintained � Output 5.1 roads extended and maintained
Performance Questions and Related Targets	Information Needs and Indicators	Baseline Information: Requirements and Status (If Known)	Data Gathering: Methods, Frequency, Responsibilities	Planning and Resources: Forms, Planning, Training, Data Management, Expertise, Responsibilities	Information Use: Analysis, Reporting, Feedback, Change Processes, Responsibilities
How has the road infrastructure improved as a result of project interventions? 150 km of main roads and 200 km of secondary roads	Kilometres of new main roads constructed per year	N/A	Record-keeping from finalisation of construction contracts		Review of road construction programme during annual project review
	Kilometres of secondary roads constructed per year	N/A	Record-keeping from finalisation of construction contracts
	Change in driving time between key locations	Driving times at start of project	Interview with key informants
	Change in road use	Level of road use at start of project	Road-use survey: baseline, mid-term, project completion, three years after completion	Design of survey