 |
|||||||||||||||||||||||||||||||||||||||||||||||||||
|
The aim of IFAD and its partnerships is to enable the rural poor to overcome poverty. This puts social mobilisation and participation of the poor upfront in conceiving and implementing development initiatives. If they are not, then relevance and sustainability of impact will be compromised. The collective experiences of partnerships between the poor, the state, NGOs, and funding organisations need to be continually assessed, in order to take systematic corrective action. This is the prime purpose of monitoring and evaluation. This Guide is about using monitoring and evaluation (M&E) to improve the impact of IFAD-supported projects. The focus is on a learning approach to M&E that uses achievements and problems for better decision-making and accountability. It requires creating an M&E system that helps primary stakeholders, implementing partners and project staff learn together in order to improve their development interventions on a continual basis. Because the ultimate objective is to ensure the maximum possible benefit for the rural poor, they are the ones best placed to assess project impact. The Guide suggests ideas for implementing this and other forms of participatory M&E. M&E is a management tool for those who manage anything from a small project component to an entire project. Setting up a good M&E system requires careful thinking about overall project management and, particularly, how to manage the linkages between different project elements and partners. Therefore, the Guide focuses on practical ideas that can help to manage for impact. Many of the issues faced in project management or when setting up a useful M&E system are affected by the original project design. So, the Guide deals with good project design and management practices - but only from an M&E perspective. The Guide will help project implementers, including primary stakeholders:
In a nutshell, this is what M&E is all about. The Guide has been written for four main audiences: Due to the many audiences, the Guide has been structured to provide readers with different levels of detail appropriate to their needs. To know which section is appropriate to the tasks you face, see "Navigating the Guide", at the beginning of the Guide and Section 1.3. People often feel overwhelmed and confused by M&E due to the many ways to undertake it and also because it is often assumed that anyone can "just do it". This Guide recognises that M&E is a professional field in its own right. Indeed, people are not expected to be agronomists, veterinarians, irrigation engineers or accountants when they have no training and no experience. Yet, it is often unfairly expected that anyone with minimal support should be able to do a good job at M&E. The Guide aims to provide both the key concepts and the practical details needed to make M&E work. The ideas in this Guide are not a mandatory M&E system with which all projects must comply. The Guide describes what is considered - and has proven to be - good practice in project M&E, with examples from experiences in many different contexts. You will not find, for example, a set of common categories of impact or fixed sets of indicators or a list of indispensable methods. Having options is critical, as each IFAD-supported project is unique. Nevertheless, good M&E does need to meet a minimum set of requirements and standards. This Guide will discuss these requirements and standards, while indicating where options are possible. No document, including this Guide, can hope to solve all problems of inadequate M&E. Other supporting measures are needed, including training, technical assistance, incentives and adequate resource allocation. In particular, improvement-oriented critical reflection is needed by those involved. These topics are discussed in detail in Sections 7 and 8.
1.2.1 Enhancing and Understanding Impact Just what is "impact" in rural development? IFAD defines impact as changes - positive or negative, intended or unintended - in the lives of the rural people as they and their partners perceive, as well as sustainability-enhancing change in their environment, to which the IFAD-supported project has contributed. Impact is often used to refer to the highest goal-level achievements of a project, such as "improved food security" and "increased household income". However, any significant effect on poverty takes several years to emerge, longer than most IFAD-supported projects. Accordingly, IFAD uses impact in a broader sense to refer to a wide range of observable changes that help reduce poverty. For example, "adoption of improved farming techniques" is an important intermediate impact. So are "building linkages between local committees, fishermen's associations and formal management bodies in ways that have ensured their participation in decision-making processes affecting their wellbeing". Impact is clearly a broad concept. Local ownership and building capacity are often critical interim impacts that encourage self-management for development amongst the poor. So is the reduction of vulnerability. A main cause of poverty, vulnerability is not just about food insecurity or the inability to meet basic needs. It concerns people's inability to influence decisions affecting their lives, negotiate collectively for better terms of trade and services, stop corruption and violence, and make organisations - government or non-governmental, public or private - accountable to them. Impact has many faces. The key idea in this Guide is to use M&E to help manage the resources and activities of a project to enhance impacts along a continuum, from short term to long term. This requires clarity in the project about the desired goals, and vigilance to understand if lower-level outputs are contributing to higher-level goals ones amidst other influencing factors. It also requires conscious effort to learn to identify corrective actions that could further enhance impact. 1.2.2 Supporting the Rural Poor Working on the details of project management, design and M&E may sometimes push the rural poor to the background. But remember that this is the prime reason why development interventions receive IFAD funding. More than 1.2 billion people live in extreme poverty - one out of every five people. It is the 900 million of these who live in rural areas that IFAD funding aims to help. Project staff and partners can claim success when society's most marginalised people themselves indicate how they have benefited directly. To enable this, M&E must build on existing inspiration, creativity and motivation. This means creating opportunities for the poor to make their own judgements about the value of projects and how to improve them. Good M&E can provide evidence of general impact. It can also capture human stories of personal change that are needed for governments and their constituents to continue supporting rural development. Keeping track of the details of project implementation and gathering good information about what has been achieved are very important. But in the end, what makes the difference is how people interact, how ideas are shared and developed, and - in so doing - how people are motivated and supported to learn and contribute to the best of their ability. Rural people's aspirations and their own development processes must be at the heart of managing for impact and M&E. 1.2.3 Increasing Participation In May 2000, an IFAD workshop on impact achievement stated, "participation means more than just beneficiary contribution to project execution, rather that it should encompass all the stakeholders and be formalised at all stages of the project cycle." This clearly includes M&E. IFAD-supported projects entail varying degrees of participation. So, developing participatory M&E systems means that, once the basics of M&E are understood, participatory M&E is defined and ways are worked out to introduce it. This is a basic principle of the Guide. Generally, widespread recognition of weak M&E systems has led to a search for alternatives that are more inclusive of primary stakeholders. For example, project staff in Latin America had such a strong feeling that their monitoring system was performing only a control function that they discarded the entire system midway through the project. They developed a new M&E system based on the principles of participation, decentralisation and flexibility. In another project, in southern Africa, contracts between communities and the project formed the basis of transparency about the rights and responsibilities attached to participation - including M&E. The Guide describes many aspects of participatory M&E. Participation applies not only to primary stakeholders. It means giving more space to grassroots organisations, banks and other private enterprises, and others, as implementing partners. The role of project management is one of facilitation between diverse partners, rather than one of steering from a central decision-making position. Each stakeholder group has its own information needs and ways of working. This includes providing feedback to funding agencies. Such information does not have to be collected, analysed and fed back by primary stakeholders. An M&E system run entirely by and for primary stakeholders is not sufficient to meet all project needs. The M&E system is for all stakeholders. 1.2.4 Improving Management by Strengthening M&E M&E is not - and cannot be - a substitute for good project management. For M&E to succeed, it needs to be driven by managers' needs for information, their use of the information and their desire to create a learning environment. M&E in any project will therefore only ever be as good as the quality of overall management. Yet M&E is indispensable for good management. The Guide presents how M&E can fulfil this function. While the Guide touches on the broad idea of managing for impact, it is not a general guide to all aspects of project management. Inadequate M&E has two consequences: limited learning by implementers about the project's progress, opportunities and problems; consequently, the limited ability of those involved to correct operations and strategy, leading to sub-optimal impact on poverty reduction; unclear impact performance, so limited accountability to funding agencies and to primary stakeholders of projects in terms of their stated goals. As one project consultant put it, "If there is no method for tracking activities or problems or the impact of activities, how can projects justify their existence?" A collection of IFAD studies on project experiences with M&E highlighted a series of problems (see Box 1-1). Some problems have external causes that lie beyond the control of the project and restrict project activities, such as disasters or institutional environment. Others have conceptual causes, which revolve, for example, around diverging and unclear perceptions of M&E, methodology and analysis. The rest have operational causes due, for instance, to insufficient personnel or the (non-) integration of M&E by project staff and other stakeholders.
M&E feeds the whole process of assessment both of change in the lives of the poor and of the performance of IFAD and other stakeholders in relation to their obligations, functions and relationships. It looks not only at the specific project or programme but also contributes to advocacy, policy dialogue and updating understanding on poverty and its reduction. Effective M&E can:
Because this Guide has been written for all those people who have specific yet different M&E-related responsibilities and tasks, it has been produced as a set of "modules" or sections (see Box 1-2). This makes it possible for you to focus on the material that is relevant for your needs at a particular point in time. You might want to copy parts of the Guide on particular M&E functions and share them with colleagues and partners. The Guide contains five tools to help you find your way around: 1. a detailed table of contents (see beginning of Guide), plus a section-specific table of contents inside each booklet; 2. a road map of the eight core Sections of the Guide (see Figure 1-1); 3. an audience-specific list of recommended sections for specific M&E responsibilities (see "Navigating the Guide" at the beginning of the Guide); 4. a detailed index at the end of the Guide; 5. the first pages of Sections 2 to 8 are a summary of the section and can help you decide what part of the section is most relevant. Figure 1-1. A road map of the core sections of the Guide
1.3.2 Making this Guide Work for You Can a book of good M&E ideas really make a difference? No. Written words alone can never change practice. It is the people using the ideas who will make the difference. In other words, this Guide is not a guarantee that all project M&E will proceed smoothly. More is needed than a simple reading. You - as manager, M&E coordinator, consultant, cooperating institution or IFAD staff member - will need to take the ideas and adapt them to your own contexts, competencies and concerns. This will inevitably require a period of trial-and-error, the hallmark of an adaptive project. A new project faces the largest task of all. Every aspect of the learning process has to be conceived in both general and operational terms, before being tried out. New projects, as well as ongoing ones, should not be discouraged by descriptions of good and comprehensive M&E systems. Small steps can be made in triggering and improving project learning - with significant results - before a fuller system is in place. Thus the Guide offers simple options for those taking first steps, alongside more sophisticated options for those who are further down the M&E path.
In terms of managing for impact and setting up the M&E system, any IFAD project will be influenced by three aspects: the project cycle, the main stakeholders and their inter-relationships, and the relationship of the project to government processes of the lending country. 1.4.1 The Project Cycle for IFAD-supported Projects "Project cycle" is the name for various steps in the life of a funded intervention - from the initial idea to the phasing out of operations and the final evaluation. All IFAD-supported projects follow a particular cycle of steps (see Figure 1-2). Several of these happen before the project is even operating. Figure 1-2. The Project Cycle for IFAD-supported Projects
A rural development project is located in a web of relationships between many different stakeholders. Most IFAD-supported projects deal with eight key groups: local people, people's organisations, project management, implementing partners, cooperating institutions, responsible government departments, consultants and IFAD staff. These relationships need to be established, understood and managed well so that each stakeholder group can make the best possible contribution. Primary Stakeholders - the Local People Local women, men and children are pivotal to a project and its learning process. They are the primary stakeholders as their needs are the focus of the project and their views on impact are what count. This is a very diverse group and most projects specify target groups, such as "marginalised farmers", "smallholders" or "the landless" in the project area. Local people are increasingly acting as full partners in project initiatives, rather than passive beneficiaries. Most projects aim to strengthen self-reliant development, so they seek local participation in project design and implementation and assessment of the findings. If project M&E builds on existing communication and learning processes, it can enhance and enrich these. Grassroots Organisations Grassroots organisations, at community and higher levels, are important partners. They provide invaluable insights on priorities and appropriate processes during the design phase, and undertake some of the implementation of the project and/or M&E. One of their most valuable roles is in facilitating participatory processes during implementation. Project management works with grassroots organisations to create opportunities for local people to participate meaningfully in M&E activities, such as through participatory baseline studies, local impact assessments or annual project reviews. Working with them increases local ownership of the project and thus the likelihood of a sustained impact. Project Management Project management is the organisational pivot for implementation. Each project organises management in its own way. One project might have only five staff in the management unit and with most management functions (including M&E) decentralised to implementing partners. Another may have a large, more centralised management and implementation unit with, for example, 20 M&E staff. Local community members can be active in project management. Project management is responsible for ensuring that the project as a whole has clear and relevant plans, reviewing and approving work, and ensuring financial flows and reporting. The project achieves its intended impacts if management adequately supports the implementing partners to deliver quality work. The project director and M&E coordinator are responsible for establishing and operating the reflection and learning processes and for reporting to supervising bodies, funding agencies and local people. Implementing Partners Projects are implemented not only through grassroots organisations but also through government services, non-governmental organisations (NGOs), and commercial operators, such as banks. In more participatory projects, these groups often have a catalytic and advisory function since decision-making lies with the primary stakeholders. Project management may ask for bids before selecting implementation partners, or these partners may be already specified in the appraisal report. Partners are guided by contracts on their responsibilities, standards of work and style of operation. All partners are responsible for monitoring the activities they implement. Sometimes evaluations are subcontracted to assess longer-term impacts or to institutionalise annual participatory impact assessments. Cooperating (or Supervising) Institutions Once a basic project design is in place and the responsibilities of government agencies are clarified, a cooperating institution (CI) is contracted. Its role is to supervise the loan process and provide technical and financial support to the project during implementation. It should also provide methodological M&E support. CIs report project progress, problems and recommended actions to the funding agencies, including IFAD. Common CIs are UNOPS (United Nations Office for Project Services), the World Bank and regional development banks. For some projects, IFAD does the supervision directly. CIs may also be co-funders. Responsible Government Ministry Each project is placed under the responsibility of a ministry, which is often the Ministry of Agriculture or Finance (or equivalent). This ministry is the loan holder and is frequently the seat of the steering committee for the project if one is created. It often appoints the project director in consultation with IFAD. It is usually a co-funder of the project. The responsible government ministry does not always play an active role in project M&E, but receives all project reports. Key ministry officials will need to agree on any significant changes to a project's strategy, should this option emerge during an MTR/MTE (mid-term review or evaluation) or as a result of a supervision mission. Project-based M&E is critical for any feedback to policymakers. Consultants/Technical Advisors Most projects use several externally contracted consultants or technical advisors at various moments. These consultants, in discussion with IFAD, design the project and thus greatly influence its focus and mode of operation, including laying down the basis of project M&E. At start-up, they also often play a key role in designing various aspects of the M&E system and related capacity-building efforts. Consultants are contracted by IFAD for MTRs/MTEs and often for interim evaluations. IFAD Most project directors will know at least two people in IFAD: the country programme manager (CPM) and the Office of Evaluation officer responsible for the country in which the project is located. Contact is mostly with the CPM, both directly and via the CI. The CPM guides the project through IFAD internal procedures up to implementation, and facilitates all relationships during the project life, including consultant contracting. The CPM's role in M&E lies in overseeing the quality of the project design, including the M&E plan. The CPM seeks ways to embed IFAD's priorities during implementation, such as participation of primary stakeholders, and often organises MTRs or interim evaluations. CPMs participate on IFAD's behalf in the project management partnership. For some projects, they are involved in direct supervision. 1.4.3 The Specific Importance of Government Processes Of particular influence on project partnerships and processes is "the government". It is to governments that loans are given, with governments that ideas are developed, and mainly through governments that implementation takes place. Governments are the agencies responsible for impact achievement of the loan, are partners in design, and are very often the home of project management. Three levels of government are critical to most IFAD-supported projects: 1. national level: for COSOP, project design, loan negotiations, and loan repayment; 2. regional level (district, province, state): for supporting implementation with management decisions and memorandums of understanding with government organisations operating at a regional level; 3. local level: for implementation by agencies in local government. Besides these different levels, each project is likely to deal with various ministries, as most projects are active in several sectors. Special project steering groups may be created within the government system to coordinate actions and policies related to the project, although membership may include others, including primary stakeholders. Working with government as a critical partner means dealing with its capacities, limitations and the politics to which all governments worldwide are subjected. Many projects must handle frequent policy changes and staff discontinuity over time. Some may be faced with the challenge of different levels of government working with different policy priorities. Policy changes can create havoc for projects if, for example, the mandate of a key department changes. Project management is rarely an autonomous unit but often operates in a ministry with its own specific ways of working. This can create considerable tensions if the lines of responsibility and those of decision-making power are blurred or diverge. From an M&E perspective, government may require integrating the project monitoring and learning systems with those it currently uses. If these are inadequate, then it may require efforts to set up M&E that extend beyond the project.
The thinking behind development intervention is constantly evolving. Many projects used to focus on expert input to design infrastructure- and technology-development oriented projects, often with little input from primary stakeholders. Over time, attention has moved towards more participation of primary stakeholders in project design and towards strategies that build capacity and empower people to direct and manage their own development ideas. Such changes require all stakeholders to accept the consequences of participation, including uncertainty, politicisation and shared decision-making. The idea of blueprint planning has given way to more flexible, process-oriented and adaptive approaches to project implementation. More recently, the trends of decentralisation and privatisation have led to more dispersed models of project implementation. This means less emphasis on centralised project management units, with implementation being managed by primary stakeholders and private contractors. These changes have five significant implications for M&E: 1. M&E can play a role recognised to help and empower local people to control their own development, with critical self-evaluations of collective experiences reinforcing their capacity for self-management. M&E can thus direct information systems not only upward but also downward; 2. more adaptive and flexible approaches to project implementation actually require better M&E systems, as the whole model is based on being responsive to feedback from primary stakeholders and to changing circumstances; 3. monitoring and evaluating capacity development and empowerment-oriented initiatives require different approaches to M&E than assessing infrastructure development or technology transfer; 4. in a decentralised and privatised context, attention needs to be given to building M&E capacity within the implementing partner groups, rather than just focusing on M&E in a project management unit, thus making M&E a vehicle for addressing questions of governance. However, sub-contracting M&E functions carries risks for corrective actions if it dilutes a project's access to critical information and capacity for reflection; 5. the importance of downward accountability and stakeholder participation - particularly of primary stakeholders - in developing, implementing and improving the M&E process becomes essential. IFAD-supported projects encompass a broad spectrum of approaches to development, from the more conventional to the more innovative. The Guide has been written from the perspective of an approach to M&E that emphasises stakeholder participation, critical reflection for learning and flexible implementation. These ideas have been presented in a way that makes them applicable to a wide range of IFAD-supported projects, whether new or existing, conventional or innovative.
Further Reading IFAD. International Fund for Agricultural Development. General website at: http://www.IFAD.org. Also a folder on IFAD's M&E strategy, including information on its Office of Evaluation and Studies at http://www.ifad.org/evaluation. It is also possible to access IFAD evaluation reports on-line, according to country (programme or project) as far back as 1981, at: http://www.IFAD.org/list_eval.asp. PREVAL. Programa para el Fortalecimiento de la Capacidad de Seguimiento y Evaluación de los Proyectos FIDA en América Latina y el Caribe. Comprehensive Spanish website on M&E for IFAD-supported projects in Latin America. Website address: http://www.preval.org
Download
PDF Version (279 KB)
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
