The purpose of Thematic Evaluation or Studies (TE/S) is to examine IFADs experience with specific aspects, themes and processes of IFADs operations and policies. These studies are designed to assess the effectiveness of IFAD operational policies, processes, and practices. TE/S synthesise and analyse the Funds accumulated experience across countries and regions and draw crosscutting lessons. The objective of TE/S is to provide the building blocks for revisiting or reviewing existing policies or formulate new ones.
TE/S will be undertaken based on findings from project as well as Country Programme Evaluations, supplemented by further investigation, including staff fieldwork and surveys.
New TE/S Process:
In consultation with core partners, we identify and define the expected outcomes for the TE/S. Our approach is practical, applied, and based on the assumption that the degree to which our evaluation outcomes are relevant, they will be used to strengthen operational policies and procedures. For this reason, we consider it fundamental to collaborate with our partnership in defining what they consider the critical outcomes towards which the evaluation process drives.
The new TE/S process is based on the New Evaluation Process (NEP) defined in the preceding pages. Although the steps are very similar to the standard NEP, the process is adapted to the specific requirements of the type of evaluation.
During the formulation of OEs Annual Work Programme, we collaborated with our core partners to define and prioritize the TE/S for the coming year based on the following criteria.
Step 2: The Approach Paper 9
The EO will consult with others (institutions and knowledge centers) before and/or after the Approach Paper, on a case-by-case basis. The EO drafts, shares and discusses the Approach Paper with selected members of the core learning partnership (see Annex 2).
This Approach Paper will follow the basic OE guidelines and will include:
Step 3: Carrying out the TE/S
There are several activities in conducting the evaluation. The administrative requirements can vary widely, as can the type and degree of participatory methods. The typical steps include:
Step 4: Consultation
The EO discusses the preliminary results with the CLP and facilitates an agreement on next steps to be undertaken. This group will decide whether to share the evaluation results with a wider audience at this point.
Step 5: Sharing and validation with a wider audience:
If and when the core learning partnership decides to share and validate the results of its work, it may use some of the following methods:
Step 6: Completion point:
The objective of this step is to reach an agreement/understanding among the decision makers about the "building blocks" produced by the TS/TE and on the follow up measures in terms of policy development and implementation and monitoring arrangements. Depending on the nature and importance of the TE/S, and on the desired level of consensus on the outcome, the completion point could take place in the form of a final meeting with Senior Management, OSC, PD Department or Regional Divisions. It will be summarized in a written agreement/understanding regarding Lessons Learned, Recommendations and follow-up measures required.
Objectives of the Country Programme Evaluation (CPE)
Focused, results-oriented CPEs are a strategic priority for OE. They are strategic in that CPEs are expected to provide inputs for establishing effective Country Strategic Opportunities Paper (COSOP) frameworks, the importance of which as OE anticipates will increase in the future.
Our approach to the CPE is increasingly participatory. In our view, IFAD will move progressively to a COSOP process that includes a broad partnership of all relevant country stakeholders in its formulation. We have tried to design a CPE process that anticipates this kind of engagement by the borrowers and civil societies.
One essential component of any CPE is to analyse and assess IFADs experience in a given country, including the most salient features of IFADs approaches and selected characteristics of IFADs portfolio. This will be done by drawing lessons from all IFAD-financed projects in that country. CPEs are, however, not intended to evaluate projects, closed or on-going, but rather to provide - based on evaluations, supervision work, the Project Implementation Status Report and Country Portfolio Reviews focussed field work and other data collection, as required, comparative information on the most salient aspects of project performance and to develop strategic and operational orientation for IFADs future project pipeline.
The second essential component of any CPE consists of a review of the most recent information, including trends and emerging opportunities and risks affecting the rural poor in the country as well as the rural poverty profile and characteristics.
In conclusion, the main expected outcome of any CPE is a set of agreed-upon inputs in IFADs COSOP, including inputs for IFADs Policy Dialogue on rural poverty alleviation in a given country. Other expected outcomes are recommendations for the improved implementation of some of the on-going projects, as well as lessons learned at country level.
We have decided to replace the name Country Portfolio Evaluation with Country Programme Evaluation. The word programme incorporates lending and non-lending activities and better reflect the time dimension involved in the evaluation of IFADs approach in a particular country.
The CPE process is an adaptation of the New Evaluation Process and includes the main innovative features we believe strengthen its impact: the Approach Paper, interaction with a core learning partnership, and an agreement among the Partnership at the evaluation completion point.
A shortcoming was that in the past CPEs could take from between one to three years before completion, and involve a sizable amount of resources. Therefore we have designed an instrument that takes less time to complete (eight months) and uses less resources. In addition to that, we intended the CPE process to focus on the production of outcomes that can be used as inputs to the COSOP and be driven by a strong justification to be provided by our partners. Thus, the new CPE should be timely, leaner, focused on selected issues, as well as demand-driven and results-oriented. The new CPE Process engages potential users and implementers of CPE recommendations up-front, to ensure later agreements on the recommendations.
Step #1: Rationale and Justification
The CPE is determined initially during the discussion of OEs Annual Work Programme with PMD. A planned COSOP by the Regional Division is a necessary but not sufficient criterion. Another criterion is the importance of the country in the Regions overall portfolio. The availability of recent evaluation works in the country concerned will help determine the scope and intensity of the CPE. The third criterion will be the quality of the statement of justification and expected outcome provided by the regional division. A fourth criterion would be the additional value the CPM is expected to provide over the Country Portfolio Review. A fifth criterion could be the Governments direct request to undertake a CPE once the existence of this instrument is brought to their attention. Based on these considerations and others we will determine the justification for a CPE jointly with the Project Management Department (PMD).
Step #2: Preparatory Stage:
Identification of CPE issues, focus and scope. This step entails:
Step #3: Formulating and Sharing the Approach Paper.
The Approach Paper should contain the following elements:
The Approach Paper will be finalized after consultation with the Core Learning Partnership.
Step #4: Undertaking the CPE:
In conducting the CPE, the EO is responsible for a variety of activities. Some of these are administrative, while other require the use of a broad range of participatory methods, evaluation tools and facilitation skills.
Step #5: Finalizing Conclusions and Recommendations
Step #6: Completion Point
There are several levels of agreements/understandings that can be worked out with our partners: the Government, possibly the civil society groups, PMD and OE:
As with the other evaluations, the EO may use a variety of forms for documenting these agreements or understandings. It is important that the Partnership complete the process with a clear written understanding of what the evaluation outcomes are, and how they will be applied or adopted by the various members of the partnership.
9 The Evaluation Panel was not convened in the majority of cases: roughly only 71 evaluations, or about 24%, of all evaluations conducted by OE were discussed in the In-house Evaluation Panel since its establishment in 1990. The Evaluation Panel was established on an experimental basis for the following purposes : a) review evaluation reports to improve quality, b) share evaluations lessons in-house in order to for them to be included in the design of new projects; and c) decision-making on their dissemination. The Evaluation Panel also served to: (d) signal the end of each evaluation. OE believes that the learning function (objectives a and b above) of the evaluation panel in the past was limited to a relatively brief interaction among divisions in a rather formal meeting. Clearly the New Evaluation Process in general and the CPL in particular will go much farther than the evaluation panel in promoting learning through evaluations. As to the formal completion objective (d), this is included in the New Evaluation Approach by the new concept of the agreement among the parties at completion point to be prepared by the CPL. Finally, as to the dissemination of the lessons learned, OE agrees that a specific arrangement that goes beyond EKSYST/IFADEVAL is necessary. This is why OE has decided to develop a dissemination and communication strategy for the results of its evaluation work in 2000.