1. A Review of the Activities Performed in 1999

One of the Office of Evaluation and Studies’s (OE) main priorities in 1999 was to review the adequacy of the International Fund for Agricultural Development’s (IFAD) evaluation function and the contribution of evaluation activities to achieving IFAD’s corporate objectives. The review was preceded by a Users Survey and an evaluation by OE staff of their own work. Within and outside IFAD there was recognition of the quality of OE’s work and the commitment of OE staff. In addition to positive feedback from our users, there were also criticisms, to which we have agreed that we need to respond. OE recognised that overall there was a need for change to enhance the effectiveness and responsiveness of our work in trying to better meet the needs of the users of our services.

This conclusion has prompted OE to embark on a soul searching and strategy development process that has led us to define our core values, vision and mission. We have also achieved a clear definition of who the users of our services and partners are, and perhaps most importantly, developed a mind set, which places our evaluation partners and their needs in the center of our focus and attention. We have also developed clear strategic objectives to guide our work in the future, and established new strategic directions that will contribute to their achievement. OE also set its priority strategic directions for the remaining part of 1999 and 2000.¹

In 1999 we conducted 25 evaluation activities, compared to 21 undertaken in 1998. Considering the amount of work involved in the strategy development work and the conducting of a number of important additional evaluation studies not planned for 1999, this is a significant achievement. The details of these evaluations and the results achieved are provided in Annex 1.

In 1999, OE laid out a set of guiding principles for its evaluation work:

• Evaluation reports will be short, more user-friendly and produced within agreed deadlines;
• Maximise the use of participatory methodologies to involve both beneficiaries and implementing agencies in the process;
• Emphasise projects, themes and areas where the need for evaluation has been jointly identified with the Project Management Department (PMD) and co-operating institutions and where opportunities for knowledge generation on poverty alleviation are high; and
• Culminate evaluation exercises with a clear understanding among all parties concerned on the results achieved, lessons learned, required actions and follow up.

During the year, OE improved the format for its reports and emphasised the use of visual tools (maps and photographs) to render the reading more attractive. More needs to be done in this area and we recognise the importance of stepping up our efforts, in particular to shorten our report.

In 1999 we also managed to reduce the duration of the evaluation process and respect deadlines in many cases. OE has also shown itself able to take up major work, such as the evaluation of the Extended Cooperation Programme/Non-Governmental Organization (ECP/NGO) Programme, that were not included in the original 1999 programme of work. In the future, we plan to discuss up front with our partners the duration of the evaluation process, including the timing of the most important stages of each evaluation and the various deadlines. We will document this in the Approach Paper.²

All OE evaluations conducted in 1999 made use of participatory tools and methodologies to different extents. There are a variety of these methodologies currently being used in OE, therefore we have decided to embark on a review of the existing participatory evaluation approaches. Similarly, building on the results of the "Effectiveness Paper" presented to the IFAD Consultation meeting in February, we have started a comprehensive methodology review, which aims at identifying simple and cost-effective tools for the assessment of performance, sustainability and impact. Both studies were driven by OE’s resolve to take stock of the rich, but not yet assessed, variety of methodologies used in our work; the ultimate aim being the identification of the most promising good practices for the future. A similar methodological review was initiated in December on monitoring and evaluation practices within and outside of IFAD.

Progress has been made in a number of evaluations where OE and its partners have managed to reach a clear understanding on the recommendations to be adopted, the lessons learned and the follow up to be undertaken. OE would like to see this understanding among partners work as the completion point of all collaborative evaluations. We consider this fundamental to our new approach to evaluation, and we will give it priority in 2000. This is also one of the major requests the Evaluation Committee formulated during the review of its work and reporting practices to the Executive Board. This review began in June and ended in December 1999 with the adoption of the Committee’s terms-of-reference and rules of procedure by IFAD’s Executive Board.

In the past, the decision to undertake project evaluations was mostly left to individual Country Portfolio Managers (CPM) and Evaluation Officers (EO) to decide.³ The decision regarding thematic studies, programme evaluations and country programme evaluations was primarily taken at the level of director (in consultation with regional directors or senior management). This year, OE has conducted intensive discussions with CPMs, the Regional Directors, PMD, and Senior Management to define a Co-operative Evaluation Programme for 2000 and has worked out for the first time an annual work programme which contains the priorities and objectives for the year.

Finally, the formulation of the Annual Work Programme for 2000 was preceded by intensive discussions within OE to revisit its products and services and develop them further, with the aim of bringing them in alignment with our new strategy. As a result we now have:

• Refined the entire evaluation process;
• Defined a new process for project evaluations;
• Produced a new and leaner process for Country Programme Evaluations; and
• Developed a new process and instrument for Thematic Evaluations and Studies.

2.    Priorities & Objectives for 2000

The OE Work Programme 2000 is based on IFAD corporate strategy as well as the newly defined OE strategic objectives and priorities. As such it is focused, demand-driven and results-oriented. In addition to the evaluations to be conducted and the project support services to be provided, we have also defined a programme of policy, strategy and management activities for the coming year designed to continuously improve our work.

OE Priorities for 2000 are to:

• Use the newly redefined instruments and processes for project evaluations, thematic evaluation/studies and CPEs and review the first experiences made with these tools;
• Develop a dissemination and communication strategy for the outcome of our evaluation work;
• Define a set of consistent methodological good practices for performance-oriented participatory evaluation and M&E support.
• Define the organisational set-up of OE that is required to support the implementation of the new OE strategy developed in 1999.

3. The New Evaluation Process

New Evaluation Process-NEP Identify & select evaluations, based on strong justification and clear expected outcomes. Consult with Core Learning Partnership, develop and share Approach Paper. Select, recruit, and brief consultant and resource persons. Conduct participatory evaluation/study. Core Learning Partnership validates results and determines the process leading to the completion point. At completion point: Agree on report recommendations to be adopted and lessons to be learned, as well as follow-up required.

We are using a New Evaluation Process (NEP) (see box below) that builds on what has been most effective in the past and anticipates the value of results-oriented participation and performance improvement-based learning. The year 2000 will be a year in which we will test and validate the new evaluation processes. In our efforts to transform our evaluations into more co-operative learning processes, we want to point out a few important innovations.

The Approach Paper: (step #2 in the NEP) is new for us. We have always prepared detailed Terms of Reference for our evaluation missions; however, the Approach Paper, when done in either consultation or direct collaboration with the core partnership can harness commitment to the evaluation process and its outcomes from the start. It treats the evaluation as a short project by defining objectives and final products, methods, the stages of the evaluation process and the resources required. Each new evaluation will include an Approach Paper and use it throughout the evaluation to guide the learning process to productive outcomes.

The Core Learning Partnership (CLP): We believe we can enhance ownership for the final recommendations and build commitment to their adoption and implementation through meaningful and focused participation. The CLP (step #2 in the NEP) is a key element in our participatory methodology, specifically for Country Programme and Thematic Evaluations. The CLP may consist of representatives of CBOs, NGOs, co-operating institutions, PMD, government representatives, project level staff and the EO. In selected project evaluations we will also experiment with this methodology, trying a variety of approaches to the CLP from informal consultation to a more formal co-ordination team and summarise our experiences at the end of the year.

The Completion Point: In the past, our evaluations ended with a publication. In the new evaluation process, they will end with an understanding or an agreement, formal or informal, to adopt specific findings and recommendations. This results-orientation represents a substantial difference in the way we approach our work. Defining the expected outcome up-front prompts us to engage key actors and implementers throughout the process. We do not believe this will compromise the rigor or quality of the evaluation outcomes. It assures that a significant number of them will be productive and lead to the desired changes in the operations, programmes and policies.

---

1/ This strategic work has been summarised in the document "A New Approach to Evaluation".

2/ See the description of the New Evaluation Process on page 3.

3/ There were though, notable exceptions such as in the Western and Central Africa Division (PA) and the Latin America and the Caribbean Division (PL), where the evaluation work programme for the year was discussed between the two divisions.