Published Version of "A Guide
for Project M&E - Managing for Impact in Rural Development"
1. While formulating its New Approach to Evaluation (NAE) in 1999, the Office of Evaluation and Studies (OE) decided, as part of that new approach, to develop appropriate monitoring and evaluation (M&E) methods and tools for IFAD-supported projects. The consensus reached in-house on the new strategic direction was duly reflected in OE's Programmes of Work for 2000 and 2001.
2. The first step in the process was to review the M&E section of most IFAD evaluation reports prepared since 1990. The overall picture was not favourable as many projects saw M&E as a policing exercise, which led to a lack of commitment, mistrust or even resistance to learning from experience. Seven key problems were identified as a result of the review:
Over the past year, OE has also made a comparative review of several major
development agencies' strategies and approaches to project-level M&E
systems. The review confirmed that poorly performing M&E systems are
not unique to IFAD, similar difficulties having been encountered by other
multilateral and bilateral organizations and non-governmental organizations
(NGOs)1. It is clear that, in most project
settings, much improvement and clarity is required.
5. In May 2000, an in-house workshop reviewed a report entitled Strengthening IFAD's Support to the Development of Effective and Efficient Monitoring and Evaluation Systems at the Project Level. In stressing the need for clearer and more operational definitions the workshop strongly recommended that IFAD's existing M&E guidelines should be updated and rendered as 'implementable' as possible in order that they could be used by all stakeholders, particularly project field staff. It was agreed that focus should be placed on identifying steps to be followed in designing M&E systems and processes for full integration into project implementation.
6. Under Objective B (iii) - Improve Impact Assessment - of the IFAD V: Plan of Action (2000-2002), it was specified that IFAD should, inter alia, "identify best practices and develop tools and guidelines for an effective monitoring and evaluation (M&E) system at the project level". This objective was also reflected in the scorecard for 2000 and included as a priority activity in OE's work programme for 2001. Furthermore, the in-house working group on Impact Achievement through the Project Cycle (IAPC) held in July 2000 recommended the establishment of more participatory M&E systems, using a logical framework (logframe) approach and focusing on impact.
7. A methodological study on strengthening IFAD's support to the development of effective and efficient M&E systems was presented to the 25th Session of the Evaluation Committee in September 2000. In November of the same year and within the framework of the IAPC working group, further discussions on the subject were held with IFAD staff, project managers, consultants and field agents.
8. All these initiatives and discussions have led to the conclusion that there is need to develop new and more practical guidelines or a resource kit to ensure that M&E has a greater impact at the project level.
9. The objective of the proposed practical guide is to facilitate the development, installation and use of effective project-level M&E systems as tools for impact-oriented management. As such, its overriding goal is to contribute to improving the impact of IFAD-funded projects, as reflected in the proposed title of the guide: Managing for Impact in Rural Development - A Practical Guide for Project M&E. This initiative is being undertaken within the framework of the IFAD V: Plan of Action (2000-2002) and complements other IFAD initiatives currently under implementation or scrutiny, such as the installation of a dynamic logical framework approach and a revised project cycle, as proposed by the IAPC working group.
The guide is to be mainly targeted at the directors and M&E officers
of IFAD-supported projects, although some sections will be particularly
relevant to project implementation partners. It will also constitute an
important reference resource for consultants contracted to provide technical
support/facilitation services to projects and for IFAD's cooperating institutions.
As M&E is heavily influenced by project design and budgeting decisions,
a module on integrating M&E into project design will be targeted to
IFAD staff and consultants involved in this phase of the project cycle.
11. In December 2000, a core group of IFAD staff and representatives from cooperating institutions, known as the Core Learning Partnership (CLP)2 , was set up to steer the process of developing the guide. The services of two consultants with considerable experience in participatory and learning-oriented M&E systems were contracted for this purpose.
12. At the first meeting of the CLP in December 2000, participants reviewed the draft terms of reference for the consultants and agreed on the form the guide would take as well as the road map and the steps involved. The second meeting of the CLP, held in January 2001, reviewed the initial conceptual framework and draft outline of the guide, which were subsequently presented to PD and OE for review and comment.
13. Between March and July 2001, necessary tools and processes for the design and use of M&E systems during the project cycle were included in the draft outline together with a number of case studies (good and bad practices). In addition to the main authors of the guide, a number of other consultants have been recruited to prepare case studies on selected projects in consultation with the Fund's regional divisions. From May to September 2001, visits were made to 30 projects in 16 countries (Armenia, Bangladesh, Benin, Colombia, El Salvador, Ghana, Guatemala, India, Indonesia, Mali, Morocco, Peru, United Republic of Tanzania, Uganda, Venezuela and Yemen). Meetings were held with other project managers during two regional meetings in Asia and Latin America.
14. A first draft of the guide was reviewed by the CLP in late July 2001, followed by a workshop to discuss the preparation of a second draft for review in October. The second draft was completed in early October and distributed both within IFAD and to a number of external reviewers who were invited to attend a workshop scheduled for 30-31 October. Apart from IFAD staff, the workshop was attended by 11 managers and M&E officers of IFAD-funded projects and programmes, ten consultants and resource persons, five project supervisors from cooperating institutions and staff of FAO, the World Food Programme and the Swiss Agency for Development and Cooperation. The objectives of the workshop were to:
15. The 'prototype' is expected to be finalized during November 2001. However, a draft version (in English only) will be tabled at the Twenty-Ninth Session of the Evaluation Committee, to be held in December 2001.
16. While numerous guidelines and handbooks on M&E have been produced, almost none of them have achieved the desired result. IFAD believes that the new guide will be more effective than others and that it will be of real use in IFAD-supported projects for the following reasons:
18. M&E in rural development means learning about achievements and problems in order to improve implementation and thus achieve a better impact. The guide will offer ideas on how to set up and improve such M&E systems and processes, and it will discuss key aspects that a project management team must bear in mind in developing a sound learning system that will help to improve management and ensure accountability. It will include many practical examples of how project staff have successfully strengthened learning systems in order to achieve an impact.
19. In particular, the guide will respond to a growing number of requests from project staff, funding agencies and project beneficiaries for practical ideas on: assessing impact, rather than measuring only outputs or activities; and being more learning/lesson-oriented, and not merely driven by accountability, to ensure greater involvement of beneficiaries in M&E.
20. The ideas put forward in the guide will not constitute a mandatory M&E system that all projects must comply with. Rather, it will describe what are considered - and have proved to be - good practices in project M&E.
21. Although it is essential that there should be flexibility in taking up options, good M&E must meet a minimum set of requirements and standards. The guide will discuss these requirements and standards, and indicate where options are possible.
22. The guide is being developed as a series of stand-alone sections of two different types: (i) 'how to' sections, e.g. how to design an M&E system and how to conduct an annual project review; and (ii) reference sections, e.g. M&E methods and tools, detailed examples of M&E plans and logframe matrixes. The sections will be assembled in separate binders. (An Internet-based electronic version may also be developed to facilitate the tailoring of the guide to the needs of the various countries and regions.)
23. At the present stage, i.e. before the findings of the October 2001 review workshop have been finalized, the contents of the draft guide may be summarized as follows.
Part I - How to Make M&E the Basis of a Learning Project
24. The first part comprises nine, 15-25 page, sections.
25. Section 1 - Introducing the Practical Guide. The key messages in this introductory section are as follows:
26. Section 2 - Using M&E to Manage for Impact: An Overview. This section shows how M&E can help a project to learn about its work, and how it can guide a project to achieve maximum impact on poverty reduction. It is pertinent to all potential users - project managers and M&E officers, IFAD staff and consultants and supervisors from cooperating institutions - and explains the four key elements of impact-oriented management:
27. Other key messages are as follows:
28. Section 3 - Linking Project Design, Annual Planning and M&E. The focus of this section is on project design and its relationship with annual planning and M&E, demonstrating how to make the best use of the logical framework approach. The main audience is intended to comprise project managers, M&E officers, consultants and facilitators and project designers. Since the guide acknowledges that project (re)design is an ongoing process throughout the life of the project, managers and implementing partners must understand the principles of good design to be able to adapt project strategy and operations in response to changing circumstances and lessons of implementation experience. The key messages here are:
29. Section 4 - Setting Up the M&E System. This section shows what needs to be done in order to create and initiate an operational plan for a comprehensive M&E system. The key messages here are:
30. Section 5 - Deciding What to Monitor and Evaluate. This section, which will be relevant to all users, shows how to choose what to track, document and analyse and with whom. Identifying the information needed, for what purpose and by whom is critical to the success of any M&E system. The key messages here are:
31. Section 6 - Gathering and Managing Information. This section focuses on how best to gather information from the field and use it for good project management. The main audience of this section comprises M&E practitioners, project managers and implementation partners. The key messages here are:
32. Section 7 - Reflecting Critically to Improve Action. This section explains what can be done to ensure sufficient and appropriate reflective learning during the lifetime of a project. M&E data will only help the project if it is reviewed at meetings with the stakeholders involved.
Section 8 - Communicating and Reporting M&E Results. This section
describes what needs to be communicated as part of the M&E process,
how and to whom. The main audience here are project managers, M&E
officers, responsible ministries and IFAD and cooperating institution
staff. (This section, which was suggested by the CLP, has not yet been
written. However, it will contain information of fundamental importance
to the ministries concerned, IFAD and its cooperating institutions.)
II - Detailed Reference Material
35. Part II provides examples and background material to support the practical application of the ideas and principles presented in Sections 2-9 of Part I. It contains eight sections.
36. Section 10 - Glossary of M&E Concepts and Terms. This section, which explains what M&E words mean, is useful to all users.
37. Section 11 - Checklists for Project Managers, Cooperating Institutions and Country Portfolio Managers (CPMs). This section summarizes the core tasks of key actors in project M&E.
38. Section 12 - Methods for Monitoring and Evaluation. The main targeted audience here are the staff of M&E units and consultants. This section presents 50 methods for undertaking specific M&E tasks, and summarizes the purpose, the 'how to' and the strengths/limitations of each task.
39. Section 13 - Annotated Example of a Project Logframe Matrix. This section gives practical examples of M&E experience in IFAD-supported projects. The examples are accompanied by comments on the improvements that were made to projects. Illustrations are provided of how to improve a project logframe and avoid the most common mistakes.
40. Section 14 - Annotated Example of an M&E Matrix. This section, which applies good logic for tailoring the M&E mechanism to a practical example from an IFAD-supported project (as in Section 13), gives examples of evaluation questions, indicators and monitoring mechanisms that can inspire project actors to develop their own. The main audience here are participants in the design and start-up stages of the projects.
41. Section 15 - Indicative Table of Contents of M&E Strategies and Plans in Project Reports. This section lists what it would be useful to include in the strategic and operational M&E plan. The main audience here are project managers, M&E officers, supervisors and technical assistants/consultants.
42. Section 16 - Sample Job Descriptions and Terms of Reference for Key M&E Tasks and Actors. This section describes what should be remembered when writing up the terms of reference for critical M&E tasks/events. The main audience here are project directors and M&E coordinators.
43. Section 17 - Bibliography and Further Reading lists other reading materials available on the issues discussed in the guide.
44. The October 2001 review workshop is expected to result in a substantial improvement in the contents of the guide based on users' needs and expectations. In 2002, the prototype will be tested, in both ongoing and new projects in the different regions. It will be translated and customized according to different users and contexts. This customization phase is essential: (i) to further improve the 'practical' nature of the tool and ensure ownership by local practitioners; and (ii) to facilitate the development of new partnerships and networks of resource persons, facilitators, trainers and other service providers who are needed and requested by many projects.
45. No practical guide or handbook can in itself make a difference, however. To increase the ability of projects and programmes to achieve greater impact, a range of other complementary efforts will need to be undertaken at the field level, with and between IFAD, cooperating institutions and governments.