Part One: Policy Framework for IFAD's Independent Evaluation

The policy framework describes the purpose of independent evaluation and its stakeholders, the evaluation principles and the operational policies to be used by IFAD
for its independent evaluation work.

I. Purpose of Independent Evaluation and its Stakeholders

A. Purpose and Role of Independent Evaluation in IFAD

IFAD sees evaluation as an important contributor to its strategy for rural poverty reduction. The Strategic Framework for IFAD 2002-2006 identifies IFAD’s mission as enabling the rural poor to overcome their poverty. It elaborates on this, stating, “Poverty reduction is not something that governments, development institutions or non-governmental organizations can do for the poor. They can forge partnerships and help promote the conditions in which the poor can use their own skills and talents to work their way out of poverty.” IFAD’s catalytic role in poverty reduction is also of great importance, as reflected in the emphasis the framework gives to advocacy, policy dialogue, learning and the dissemination of knowledge. Independent evaluation contributes to IFAD’s strategy by bringing an independent perspective to the assessment of progress in relation to IFAD’s mission and catalytic role, and contributing feedback for learning.

The main purpose of the independent evaluation function at IFAD is to promote accountability and learning in order to improve the performance of the Fund’s operations and policies. Evaluations provide a basis for accountability by assessing the impact of IFAD-supported operations and policies. They are expected to give an accurate analysis of successes and shortcomings, i.e. “to tell it the way it is”. This feedback helps the Fund improve its performance. Accountability is thus a key step in a learning process that, if followed through in partnership with those who are being evaluated, deepens IFAD’s and its partners’ understanding of the causes of and solutions to rural poverty. IFAD uses this knowledge to develop better pro-poor instruments and policies to enable the rural poor to empower themselves and overcome their poverty.

IFAD’s evaluation approach reflects and is harmonized with internationally accepted evaluation norms and principles.¹ It also takes into account the specific features that make IFAD different from most other development agencies, in particular, the evolving but not yet fully effective system of self-assessment of IFAD operations and IFAD-supported projects, the absence of a field presence and the limited resources available for project supervision and learning from operations. This defining logic has various implications for the independent evaluation function at IFAD. In particular, and perhaps more so than central evaluation offices elsewhere, OE must ground its evaluation in extensive fieldwork and generate much of the evaluation-based knowledge that IFAD requires to learn from past operational experiences.

Furthermore, OE provides guidelines and technical inputs for enhancing the capacity of IFAD operational units and IFAD-assisted projects to undertake self-assessment. These inputs are rendered through evaluation of the self-assessment system and through technical advice aimed at improving the system. In relation to the latter, OE, together with other partners in and outside IFAD, has developed A Guide for Project Monitoring and Evaluation: Managing for Impact in Rural Development and provides assistance in customizing this for different regions and countries.

B. Evolution of the Evaluation Function at IFAD

IFAD established an evaluation function shortly after it began operations in 1978. At that time, however, evaluation was combined with monitoring as part of the Monitoring and Evaluation Division, which reported to the Assistant President, Economic Policy Department. In 1994, as a result of recommendations made by the rapid external assessment of IFAD during the negotiation of the Fourth Replenishment of IFAD’s Resources, the evaluation function was separated from monitoring and a unit independent of operations, called the Office of Evaluation and Studies, was established. The Director of OE started reporting directly to the President, and OE was then incorporated into the Office of the President.²

In accordance with the evaluation policy proposed in this document, OE will now operate as an IFAD organizational unit that is independent of IFAD management in the conduct of the evaluations that it undertakes.³ The OE Director will be directly responsible to the Executive Board, who will oversee OE’s work. The Executive Board has established its own Evaluation Committee to assist it in considering evaluation issues. Furthermore, OE will be renamed the Office of Evaluation.⁴

C. Evaluation Stakeholders

IFAD recognizes that evaluation has a number of important stakeholders with a range of perspectives and expectations. Stakeholders include:

(i) IFAD’s Executive Board, which represents Member States, approves the allocation of IFAD resources with the expectation of achieving a tangible and measurable impact in terms of rural poverty reduction, and will now oversee OE’s independent evaluation work.

(ii) The rural poor, for whom the success or failure of IFAD-supported projects and programmes has the most direct and long-lasting implications.

(iii) Stakeholders whose performance in managing IFAD-assisted operations and carrying out IFAD policies is evaluated by OE, namely:

• IFAD operational divisions, grouped under the Programme Management Department, and IFAD management concerned with corporate-level policies and strategies;
• Member States and their project authorities that borrow funds from IFAD for rural poverty reduction;
• cooperating institutions that perform supervision on behalf of IFAD; and
• non-governmental organizations (NGOs), civil-society organizations, and organizations of the rural poor that are engaged in IFAD-assisted projects.

(iv) Cofinanciers that supplement IFAD’s resources in particular projects.

II. Evaluation Principles and Operational Policies

A. Independence

In accordance with the proposed evaluation policy, the evaluation function at IFAD will operate in line with internationally accepted principles for the evaluation of development assistance. Foremost among these is the principle that the evaluation process should be impartial and independent from both the policy-making process and the delivery and management of development assistance.

Independence is best achieved where evaluation activities are independent from managers who have an interest in showing accomplishment and good performance, or any other decision-makers for whom evaluation raises a conflict of interest. This means that the evaluation function should be separate from and not report to IFAD’s management, which is responsible for planning and managing development assistance.

A separate budget for evaluation is another important dimension of independence. In this connection, a review of development agencies’ procedures by the Organisation for Economic Co-Operation and Development/Development Assistance Committee (OECD/DAC)⁵ notes, “Access to, and control over, financial resources and evaluation programming is an important sign of independence.” The evaluation budget is closely related to the work programme; and authority to select projects, programmes and policies for evaluation and formulate the work programme is also a key measure of independence.

Authority to select evaluators and consultants, formulate and approve their TORs and manage the human resources employed in evaluation is also important as these factors affect the independence of the process and the results of evaluation. So too is the authority to revise and finalize reports after discussion with the relevant partners.

The operational policies and procedures set down in this document incorporate all of these aspects of independence:

(i) The OE Director will be directly responsible to the Executive Board; he or she will be appointed and removed only with the endorsement of the Board, and will not be eligible for re-employment within IFAD after the completion of his or her fixed term(s).

(ii) The OE Director will be responsible for devising the OE strategy and for determining ways and means to achieve it.

(iii) The OE Director, acting independently of IFAD management with the approval of the Executive Board and the Governing Council of IFAD, will be responsible for formulating OE’s annual work programme and budget.

(iv) The OE Director will have the authority to issue final evaluation reports directly and simultaneously to the Executive Board, the President and other stakeholders, and disclose them to the general public without prior clearance from anyone outside OE.

(v) The President will delegate to the OE Director authority to make all personnel and operational decisions concerning OE staff and consultants in accordance with IFAD rules and procedures.

B. Accountability

IFAD recognizes that a main purpose of evaluation is to provide a basis for accountability, including the disclosure and dissemination of information to the public. Accountability in this context refers to the assessment of developmental results, the impact of development assistance and the performance of the parties involved. This is different from accountability for the use of public funds in financial and legal terms, usually the responsibility of auditors and legal specialists.

IFAD considers accountability as a necessary first step in the learning process. Systematic independent evaluation of completed projects and past and ongoing policies and strategies is indispensable if IFAD is to learn from its experience, both positive and negative, and improve its future effectiveness.

Accountability through evaluation analysis requires a rigorous methodology for the assessment of developmental results and impacts and the performance of the partners concerned. It also requires that successes, unexpected results, shortcomings and failures highlighted during the evaluation be disclosed to relevant stakeholders and the general public without interference from any vested interest.

Moreover, the accountability of an international development organization as a whole is facilitated if the results of individual evaluations can be aggregated and consolidated at the organizational level. This allows a better analysis of the effectiveness of a given development organization and of the cross-cutting issues that impinge on its overall performance.

These aspects of accountability will be reflected in the following operational policies:

(i) As in the past, every year OE will evaluate on the basis of clear criteria a sample of completed IFAD projects, a number of IFAD cooperation strategies in countries with large IFAD portfolios, as well as key IFAD policies, strategies, programmes and processes.

(ii) Interim evaluations will remain mandatory before a further phase of a project is embarked on or a similar project is launched in the same region.⁶

(iii) IFAD management will ensure that IFAD officials and IFAD-assisted projects promptly provide all documents and other information required by OE, and participate and cooperate actively in the evaluation process.

(iv) The OE Director will issue evaluation reports to the President and the Board without prior clearance from anyone outside OE.

(v) As in the past, OE will ensure that all evaluation reports and other evaluation products are disclosed to the public at the completion of the evaluation process and disseminated widely through the print and electronic media in accordance with IFAD’s disclosure policy.

(vi) OE will work with a methodological framework for evaluation that helps assess and evaluate impact at project completion; produce a consolidated picture of the results, impact and performance of a cohort of projects in a given year; and synthesize learning from evaluation.

(vii) This methodological framework will be the basis for an annual report on the results and impact of IFAD operations, which OE will present to the Board and IFAD management starting in 2003.

C. Partnership

Establishing a constructive partnership between OE and other relevant stakeholders is essential both for generating evaluation recommendations and for ensuring their uptake and ownership. Fostering such partnership takes time and effort, and depends crucially on the attitude and behaviour of those conducting the evaluation. Meaningful partnership also requires, inter alia, that evaluations are perceived by stakeholders as being useful, well informed, relevant and timely, and are clearly and concisely presented. Given the value of partnership, OE intends to make respect for the partners whose performance it is called upon to evaluate a main starting point of its evaluation work.

IFAD evaluation policies and instruments aim at the appropriate engagement of stakeholders in the evaluation process, while safeguarding the independent role of OE. OE remains, however, solely responsible for producing the evaluation report and its findings. In particular, the following existing policies and practices will retain their validity:

(i) As in the past, at both the beginning of the evaluation process and during fieldwork, OE will invite all relevant stakeholders, including the operational staff of IFAD and the borrower country, cooperating institutions and beneficiaries, to contribute information and insights.

(ii) At the beginning of every evaluation, OE will continue to ensure that the evaluation process is understood, is transparent to all stakeholders and includes a timetable agreed with them.

(iii) In line with international good practices in evaluation, OE will, as in the past, share draft evaluation reports with all concerned for purposes of obtaining comments, in particular on possible factual errors and inaccuracies.

(iv) To firm up the partnership aspects of evaluation mentioned above, OE will, as before, form a core learning partnership (CLP) among the main users of the evaluation.

D. Learning

Establishing effective feedback loops from evaluation to policy-makers, operational staff and the general public is essential if evaluation lessons are to be learned. The recognition that feedback that stakeholders understand and find useful is a key output of evaluation has led OE to nurture partnerships with stakeholders to ensure that evaluation recommendations are adopted and lead to the required changes and performance improvements. OE recognizes, in particular, that often evaluation reports, by their very nature, cannot propose the kind of clear-cut operational recommendations that implementers need. Similarly, OE recognizes that evaluation results need to be communicated through user-friendly products. It will pursue this objective through the following existing policies:

(i) As is current practice, after completion of the independent evaluation report, OE will facilitate a process through which the main users of the evaluation can deepen their understanding of the evaluation findings and recommendations and make them more operational.

(ii) As and when required, the OE Director will assign evaluation officers to participate in selected project development teams and programme and policy working groups in order to facilitate the understanding of evaluation recommendations.

(iii) In addition to the evaluation report, OE will continue to prepare short, easy-to-read communication products on evaluation findings and recommendations and disseminate them widely among IFAD staff, their development partners and the general public.

While OE has a key role to play in generating and communicating evaluation lessons, responsibility for uptake and learning extends beyond OE and requires the commitment of IFAD management, the Executive Board and country stakeholders to follow up and act on lessons learned. In response to this need, IFAD will institute the following policies for establishing effective feedback loops:

(i) The President will ensure that evaluation recommendations found to be feasible by users are adopted at the operational, strategic and policy levels (as appropriate).

(ii) As in the past, the Evaluation Committee will provide feedback to OE and report to the Executive Board on specific evaluation issues, and the latter will provide feedback to IFAD management.

---

1/ As set down in the Organisation for Economic Co-operation and Development (OECD)/Development Assistance Committee (DAC), Principles for Evaluation of Development Assistance, OECD, Paris, 1998.

2/ Annex II summarizes the significant milestones associated with the evaluation function at IFAD.

3/ Operational arrangements required for ensuring the independence of OE from IFAD management will be given effect through a bulletin issued by the President of IFAD after the Executive Board decision on a new policy and as required by such a policy.

4/ It is proposed that the words “and Studies” be dropped from the name of the Office in order to describe more accurately OE’s core business and bring its name in line with the evaluation offices of other international financial institutions.

5/ OECD/DAC, Review by DAC Principles for Evaluation of Development Assistance, OECD, Paris, 1998, page 24.

6/ In the event that IFAD operations do not meet the requirements to include an interim evaluation in OE’s work programme, then it is the responsibility of IFAD’s management to provide the necessary justification to the Board.