Rural Development Project
The project performance assessment of the Rural Development Project supported rural finance, value chain and enterprise development, as well as institutional capacity-building. Although implemented in a critical transition period in Georgia (2006-2012), it effectively addressed selected bottlenecks in rural development, such as lack of financing for agricultural production, insufficient food safety services, weak agribusiness facilities and feeble land registration systems. The investment in providing microcredits to farmers was deemed as a highly relevant and effective tool in reviving agricultural and livestock production, and ensuring food security. On the other hand, this assessment revealed that the agricultural supply chain activity component was under designed, underinvested, and suffered approval delays and cancellation of some activities.
North-East Development Project
This project performance assessment is the first undertaken by the Independent Office of Evaluation in the Republic of Azerbaijan. The North-East Development Project was the third IFAD-funded initiative in the country and was implemented during the period 2006-2011, in a transitional context. The project’s successful introduction of participatory irrigation management to the benefit of 15,000 households was an impressive achievement. It successfully established water user associations and rehabilitated nearly 31,000 ha of farmland. This led to a significant rise in water supplies and improved water-use efficiency, providing a sound basis for introducing higher-value crops, increasing crop yields and expanding livestock activities. At the same time, the project’s investments in agriculture, livestock and marketing were not commensurate with the critical importance of these activities to the livelihoods of smallholder farmers.
Evaluación de los resultados de un proyecto
La República Oriental del Uruguay, ubicada en el Cono Sur entre Brasil y Argentina, tiene una población de 3 300 000 habitantes, con una densidad de 19 habitantes por km2 y es un país independiente desde 1830.
El Proyecto Uruguay Rural (PUR) fue concebido en el marco del programa sobre oportunidades estratégicas nacionales que se aprobó en 1999, como una segunda fase del Programa de Apoyo al Pequeño Productor Agropecuario, el primer proyecto financiado por el FIDA en Uruguay. En el año 2000 cuando se aprueba el PUR, comienza un nuevo gobierno cuya orientación en la agricultura y desarrollo rural era productivista, sin darle prioridad al combate a la pobreza. Combinado con la megacrisis en Argentina del 2001-2002, que repercutió fuertemente en Uruguay provocando una caída del producto interno bruto del 10,8% en 2002, y una evolución negativa en todas las variables, el PUR no fue considerado un proyecto prioritario y esto se tradujo en un bajo nivel de ejecución. Al tiempo de la Revisión de Medio Término de 2005, sólo se había desembolsado el 18% de los recursos. También cabe destacar que en el período de la evaluación hubo una seca muy importante a nivel nacional (años 2008-2009).
El monto total del préstamo aprobado en el año 2000 fue de 14 millones de dólares, de los cuales al cierre del préstamo fueron utilizados el 92%. El proyecto tuvo alcance nacional y hasta el año 2007 la supervisión estuvo a cargo de la Oficina de las Naciones Unidas de Servicios para Proyectos. A partir de ese año el FIDA se encargó directamente de la supervisión del proyecto. El objetivo principal del PUR fue la reducción de la pobreza rural, mejorando los ingresos de los beneficiarios y su calidad de vida (aspectos sobre los cuales no se ha generado información), con una estrategia que apuntó a crear un mecanismo institucional para la sostenibilidad en el tiempo de las políticas y de los instrumentos de desarrollo rural y la reducción de la pobreza en el campo.
Los objetivos específicos fueron los siguientes:
- fortalecer las instituciones del sector agropecuario a nivel central y local para asegurar la sostenibilidad de las acciones y de las políticas de desarrollo rural y fortalecer las organizaciones locales de los beneficiarios para asegurar la participación;
- asegurar y mejorar en forma sostenible el acceso a los servicios de apoyo a la producción por parte de los pequeños productores y las pequeñas y medianas empresas;
- mejorar el acceso de los pequeños productores y pequeñas y medianas empresas a los servicios financieros rurales;
- contribuir al fortalecimiento de un mecanismo institucional de coordinación de las políticas y proyectos de desarrollo rural y, por medio del sistema de monitoreo y evaluación, contribuir a acelerar los procesos de aprendizaje e innovación en materia de desarrollo rural. El párrafo siguiente muestra el resultado de aplicar al PUR los criterios evaluativos del FIDA en los que se descompone el desempeño del proyecto.
Pertinencia: los componentes del proyecto correspondieron adecuadamente al diagnóstico, y el diseño del proyecto era coherente con los objetivos. La calificación correspondiente a pertinencia es de 5 (satisfactoria).
Eficacia: si bien el PUR no consiguió alcanzar todos sus objetivos, fue capaz no sólo de llegar sino también de sobrepasar las metas en áreas clave en las que el Gobierno de Uruguay ha continuado avanzando después del cierre del PUR. La calificación correspondiente a eficacia es de 5 (satisfactoria).
Eficiencia: la información disponible sobre la eficiencia del PUR es muy limitada. Sin embargo, cabe señalar el efecto negativo sobre la eficiencia de la doble postergación de la fecha de cierre del proyecto; la dispersión de la población fue otro factor que afectó la eficiencia, junto con las modalidades de intervención adoptadas. La calificación correspondiente a eficiencia es de 3 (moderadamente insatisfactoria).
Innovaciones y escalamiento. El PUR apoyó la introducción de innovaciones institucionales (Mesas de Desarrollo Rural, Comités de Crédito Local, creación de la Dirección General de Desarrollo Rural) para la lucha contra la pobreza rural en Uruguay, lo cual en parte fue posible por la incorporación de equipos multidisciplinarios, integrados por profesionales no sólo de las disciplinas agropecuarias tradicionales sino también con especialistas provenientes de las ciencias sociales. Las Mesas de Desarrollo Rural son plataformas de articulación público-privado, que no existían antes y han permitido a las poblaciones pobres participar en diálogos con el sector público y privado para hacerles llegar sus demandas. Los Comités de Crédito Local son nuevas formas de gestión de los créditos para poblaciones rurales pobres, dando un papel central a la propia población, que trabaja voluntariamente en los Comités, utilizando su conocimiento local y movilizando el control social para colocar presión en el pago de los compromisos. Finalmente, la creación de la Dirección General de Desarrollo Rural, impulsada por el PUR, es una innovación organizativa del Ministerio de Ganadería, Agricultura y Pesca, que permite alcanzar mayor eficacia y eficiencia en las intervenciones orientadas a combatir la pobreza rural, facilitando la coordinación de intervenciones y generando oportunidades para el escalamiento de las innovaciones. Estas innovaciones han tenido un impacto significativo sobre las instituciones y políticas públicas del Uruguay para combatir la pobreza rural, con altas perspectivas de sostenibilidad. Además, cabe señalar que el PUR también consiguió avances en igualdad de género y empoderamiento de la mujer que han sido institucionalizados.
En síntesis, cuando en el año 2005 asume en Uruguay un nuevo gobierno, fuertemente orientado a combatir la pobreza y a promover la participación popular, el PUR aceleró su implementación, que había sido lenta durante sus primeros años de ejecución. El PUR contribuyó particularmente a partir de ese año a un desarrollo institucional orientado a eliminar la pobreza rural tanto a nivel micro, con la formación y consolidación de Comités de Crédito Local, a nivel meso, con las Mesas de Desarrollo Rural, que permiten la coordinación público privada, y a nivel macro, con la creación y apoyo a (y en) la Dirección General de Desarrollo Rural en el Ministerio de Ganadería, Agricultura y Pesca. Estos resultados fueron alcanzados por la combinación de un conjunto de factores: el compromiso del Gobierno de Uruguay, un diseño de proyecto que identificó adecuadamente las brechas institucionales y el modo en que podían solucionarse, y una participación activa del FIDA en la supervisión del proyecto. Tres aspectos críticos son elaborados en las conclusiones del informe: el desfasaje entre el ciclo del proyecto y el ciclo político, las deficiencias del seguimiento y la evaluación y la falta de aprovechamiento de la experiencia mundial del FIDA en financiamiento rural y asistencia técnica para las poblaciones rurales pobres. Finalmente, para cada una de estas cuestiones, el informe propone recomendaciones específicas.
Madagascar Country Programme Evaluation
This country programme evaluation is the first carried out in Madagascar and it covers the period 2000-2012, the 2000 and 2006 country strategic opportunities programmes, six loans and seven grants. Throughout the period evaluated, IFAD’s activities have been relevant and highly effective in improving agricultural productivity and diversifying rural incomes through support to enterprises.
The country programme has brought about promising results in the development of agricultural value chains and partnerships between the public and private sectors. Despite the political crisis that hit the country in 2009, IFAD was the only one of the main donors in the agricultural sector not to suspend its cooperation.
On the other hand, two issues require more attention: consolidation of project results after their closure and environmental protection (erosion control measures, soil fertility conservation and reforestation).
Rural differentiation and smallholder development
This synthesis report is based on two main sources of information:
- review of current external literature; and
- IFAD sources (a review of IOE evaluations, Programme Management Department (PMD) documentation on targeting, and IFAD policy and strategy documents.
Section II draws examples from the development literature of analyses of differences among rural populations, and potential pathways for the development of various rural groups. It then relates these to a consideration of the policies that might facilitate the progress of groups along these paths.
Section III focuses on IFAD’s experience in targeting.
The final section summarizes the key issues and IFAD’s options for strengthening its impact on poverty and hunger through approaches that address rural diversity and maximize the pro-poor benefits of rural and smallholder agricultural development.
The annex contains a more in-depth analysis of the potential of different types of rural and agricultural development policies to benefit diverse categories of rural people, and IFAD’s potential role in relation to those policies.
Mali Country Programme Evaluation
IFAD started its operations in Mali in 1982 since when it has financed 12 projects. This country programme evaluation is the second carried out in the country and it covers the period 2007-2012. Despite the adverse situation experienced in Mali due to conflicts in the north since independence and a coup d’etat in March 2012, the cooperation programme between the Government of Mali and IFAD has generally improved since the first country programme evaluation (2007). The most promising aspects include the adaptation of projects to decentralization mechanisms and institution-building among local communities. In addition, interventions with regard to rural finance have been refocused in line with principles of sustainability and more fully mainstreamed into national strategies. In terms of efficiency and sustainability, the results are mixed and have been affected not only by lower rates of achievement of physical targets than anticipated and significant increases in implementation and management costs, but also by the conflicts in northern regions.
Rural Poverty-Reduction Programme
The project was successful in providing key services to remote herders such as hospitals, school dormitories, kindergartens and literacy training. It supported income diversification, and was especially successful with vegetable and crop growing. However, overly optimistic estimates were made of the impact of the new range management monitoring committees and of what these could achieve in terms of increasing livestock productivity. So while herder incomes grew rapidly during the project life, it is dangerous to attribute this to project activities since it was a period of rapid general economic growth in Mongolia.
Rural Poverty Reduction Project in Prey Veng and Svay Rieng
The project performance assessment project had positive impacts with regards to productivity increase, livelihoods and food security, and met its overarching objectives. The significant economic and social changes that were taking place in the country during the project period helped the Government of Cambodia to achieve these results. Positive processes were initiated and have been scaled up by the Government such as the participatory planning, management and budgeting of the Commune Infrastructure Development Fund. However, technical and institutional problems with the design of roads and irrigation systems prevented the infrastructure components from benefitting the anticipated number of beneficiaries and from being sustainable
IFAD's Institutional Efficiency and Efficiency of IFAD-funded Operations
Background and objectives
This report presents the findings and recommendations of the corporate-level evaluation on IFAD's efficiency (CLEE). The CLEE responds to the growing interest in the organization's efficiency in the wake of the 2008 global economic and financial crisis, and the ensuing budget constraints affecting many IFAD member states.
The main audience of this evaluation is IFAD Management and the Executive Board. However, the evaluation will also be of interest to IFAD Member States, multilateral and bilateral organizations, as well as the development evaluation community in general. CLEE covers not only the efficiency of IFAD operations, but also institutional efficiency in a number of critical areas: the management of human resources, results and budgets, ICT, oversight and support functions, leadership and decision-making, and governing bodies. Given its scope and coverage, it has been arguably the most complex and challenging evaluation done by the Independent Office of Evaluation (IOE). The challenge has been exacerbated by the fact that this evaluation is the first of its kind among development agencies, requiring IOE to develop a dedicated methodology, and mobilize extraordinary expertise in evaluation and a range of technical areas. The unique features of this evaluation include the analysis of efficiency at multiple levels – output, outcome, impact and scaled-up outcome and impact, which is very relevant for IFAD. Credit goes to the Executive Board and IFAD Management for their support in this undertaking, as well as for their openness to engage with IOE throughout the evaluation process, and share with us the necessary data, information and documentation required for our analysis.
IOE has been ably supported by a highly credible team of consultants, with country case studies mostly done by national consultants (annex 14). IOE has benefitted from the insightful inputs of two senior independent advisers, Robert Picciotto and Richard Manning, who ensured that IOE and its consultants followed the most appropriate evaluation methodology and process to conduct this evaluation, and who also reviewed and commented on several deliverables, including the draft final report. Their joint analysis on the quality of the evaluation has been included (annex 10). The CLEE also included valuable contributions by several IOE staff at different stages, however, in line with the IFAD Evaluation Policy, as the manager of the CLEE, I take full responsibility and ownership of the contents in the final evaluation report and the overall evaluation process.
In terms of process, special effort was made by IOE throughout the evaluation to minimize surprises and maximize learning and dialogue. For example, emerging evaluation findings were discussed with IFAD Management in a timely manner, both informally and formally. Management was given the opportunity to review and comment on the emerging findings in an interim report, and their feedback was duly considered by IOE in the preparation of the draft final evaluation report, where they also had a chance to comment. The main findings were presented to the Evaluation Committee and the Executive Board for their feedback before the preparation of the final evaluation report.
The CLEE found that a number of initiatives to enhance efficiency and effectiveness have been undertaken in recent years, including in the course of the evaluation. They include the introduction of direct supervision and implementation support, expanded country presence, the Change and Reform Agenda, commitments related to efficiency undertaken in the context of IFAD9 consultations, a more transparent and strengthened budget process, and the even more recent Strategic Workforce Plan.
On the operational front, IFAD is engaged in a fundamental transition from a focus on financing individual projects to a programmatic approach that links knowledge work, policy dialogue and partnerships to projects in each country, with growing attention given to the scaling up agenda. It will take time for the full benefits of many management decisions and reforms to be realized fully. At the same time, there are a number of opportunities for increased efficiency at different levels, and they are enumerated in the report. In pursuing these opportunities, IFAD Management and the Executive Board must be mindful of the trade-offs between efficiency at these different levels.
With respect to the governing bodies, CLEE finds that their overall architecture is complex, but it works. The non-resident nature of the Board is a positive characteristic but it does contribute to an overloaded agenda. There are opportunities for the Governing Council to delegate to the Executive Board, and for the Board to delegate to Management. The report includes a dedicated chapter on the efficiency of IFAD governing bodies, with many interesting findings that deserve reflection.
Recommendations. The evaluation has a number of important findings and recommendations that merit close attention. The overarching recommendation is that IFAD "Raise the bar for IFAD's own performance as a partner to promote scaled-up impact of IFAD-supported operations". This recommendation is grounded in the rationale that for IFAD to make a significant impact, it must leverage its resources manyfold by attracting partner resources on a very large scale to replicate and scale up pioneering, innovative IFAD-funded operations and activities. This requires that IFAD become the center of excellence in its niche by raising the bar on its own performance. It compels IFAD to aspire to the threshold of ‘satisfactory or better' performance as against the current practice of ‘moderately satisfactory or better'. That is the first step toward excellence in all aspects of operations, and toward developing IFAD-supported projects and country programmes that can lead to the desired levels of scaled-up impact.
This recommendation is supported by seven sub-recommendations:
- Scaling up of high impact, innovative approaches emerging from IFAD-supported projects and programmes should become the objective of IFAD's business model;
- Articulate and implement a clear vision for country presence and how IFAD would operate in a decentralized environment;
- Manage oversight and support units, including critical ICT functions, with a clear focus on increasing service quality and cost efficiency;
- Better manage scarce budgetary resources towards high-quality results;
- Manage strategically the skills composition, cost and performance of the workforce;
- Focus oversight by the governing bodies on key strategic issues, with more attention to discussing results, lessons and evaluations;
- Instill an institutional culture of accountability and performance, and strengthen the reporting for results.
With respect to the governing bodies, the evaluation also recommends the introduction of a code of conduct, a normal feature of international financial institutions, and the development of broad terms of reference for Executive Board members to assist Member States in designating their representatives. Discussions on the CLEE by the Evaluation Committee and the Executive Board have resulted in broad agreement on the recommendations, and highlighted the critical importance of effective and timely follow-up. Management and IOE agree that the best way forward is to combine the recommendations of the evaluation and the efficiency-related commitments undertaken in IFAD9 consultations into a single, consolidated action plan for further enhancing IFAD's efficiency.
IFAD’s Engagement with Cooperatives - A Study in Relation to the United Nations International Year of Cooperatives
Cooperatives are not the only farmers' organizations (FOs) to receive IFAD support. However, because the United Nations has designated 2012 as the International Year of Cooperatives (IYC), it was considered appropriate that the synthesis report should focus mainly on cooperatives. Other forms of FOs are also discussed, not only owing to their importance but also because IFAD documents often do not distinguish between cooperatives and other types of FO.
In the context of the IYC, the Executive Board requested the Independent Office of Evaluation of IFAD (IOE) to study the Fund's engagement with cooperatives, particularly with regard to IFAD's evolving approaches to supporting cooperatives, lessons learned and good practices thatwould contribute to furtherdebate and reflection on the topic. To that end, IOE reviewed IFAD's policy and strategy documents relating to a limited but select sample of projects and grants, which revealed that IFAD has provided a wide range of financial support to cooperatives and similar organizations. A typology was also drawn up of planned and actual operations and lessons learned in designing and implementing projects and grants relating to cooperatives and similar FOs. However, to determine the full extent of IFAD's financial support to such organizations would have called for a much larger sample than the allocated resources allowed for the task.
IFAD's policies and approaches. IFAD has no stand-alone policy on cooperatives or other FOs, and the term "cooperatives" is not specifically mentioned in its strategy and other official documents. For at least the last two decades, IFAD has consistently promoted organizations with values and principles similar to those of cooperatives, and, when discussing support provided above the farm level, its annual reports or policy documents have used more general terms. From the late 1990s, and especially since the year 2000, the terms "farmers' organizations" and "rural people's organizations" appearing in the Fund's annual reports have implicitly encompassed agricultural cooperatives. In the IFAD Rural Finance Policy of 2000 (updated in 2009), the term "rural financial institutions" also encompasses cooperative savings and credit associations.
Terminology and study material. Because IFAD has not distinguished cooperatives from other FOs, and as most FOs are membership-based and democratically led, it was considered appropriate that they should be included in the study. The definition of FOs recommended by the IFAD-sponsored Farmers' Forum has been used, that is: "Farmer organizations are membership-based organizations of smallholders, family farmers and rural producers – including pastoralists, artisanal fishers, landless people and indigenous people – that are structured beyond the grass-roots or community levels, to local, national, regional and global levels." Cooperatives, which are also membership-based and democratically managed organizations, fall under this definition, as do non-cooperative, registered farmers' associations and economic farmers' associations. Local farmer groups and village development committees have not been included. Altogether, 25 projects from the five IFAD regional divisions and 10 grants, usually supporting multi-country activities, were selected for the study inasmuch as they included components in support of FOs.
Typology of IFAD's assistance to cooperatives and similar FOs. IOE reviewed the material (project and grant design, appraisal and President's Reports and Recommendations, plus mid-term review, supervision, ex post evaluation and project status reports) for information on the IFAD assistance that was planned and results actually achieved. Items of special interest were activities considered important for the development of cooperatives and other FOs, and for IFAD, and involved the inclusion of FOs in the goals and objectives of projects; components in support of such organizations; problems (challenges) identified and assistance planned and executed; governance issues; beneficiaries; gender promotion; and risk analyses regarding cooperatives and other FOs. The results of the findings are given in the main report but, for the purpose of this summary, the following aspects are highlighted.
Although establishing cooperatives and other FOs, and building up their networks and vertical support organizations, are demanding tasks, their management – especially of cooperatives – is even more demanding, particularly owing to their democratic nature and large, often poorly educated membership. The project and grant designers identified many potential problems, which the study team classified into the following broad groups:
- Inadequate organization by rural populations;
- Poor reputation of cooperatives;
- Effects of economic liberalization;
- Lack of experience and relative financial weakness of cooperatives;
- Lack of competencies and systems;
- Poor infrastructure;
- Lack of vertical integration and linkages; and
- Inadequate finance.
The measures used to deal with such problems included:
- Building up capacity (usually in the form of technical assistance, training, workshops and study visits) at all levels: for staff, committee members and ordinary members of cooperatives and other FOs;
- Strengthening institutions (helping to establish FOs and providing technical assistance or funds for basic infrastructure, equipment, and technical or managerial backstopping);
- Provision and strengthening of financial services;
- Technical assistance for special tasks and studies; and
- Supporting vertical structures for advocacy and policy dialogue with government and traders.
These measures, applied selectively in individual projects and grants, generally produced at least moderately satisfactory results. In view of resource limitations, IOE was unable to evaluate the individual projects (and, in any event, only five projects had completed post-evaluations reports). However, the mid-term and supervision reports showed that all but four1 had average ratings of 4 or more (from "moderately satisfactory" to "satisfactory"); that projects rated below 4 were, on average, deemed as above "moderately unsatisfactory"; and that, with regard to institution-building, even the four weaker projects were rated above 4. In IOE's view, these ratings would have been higher had situational (economic, social, educational, historical and even psychological situations) and institutional (needs for capacity-building and institutional strengthening) analyses been undertaken before project/programme design.
IFAD has used cooperative projects – and to some extent other FO projects – to reach large numbers of beneficiaries and promote women's participation. In many places as much as 30 per cent or more of the thousands of beneficiaries are women. Cooperative savings and credit societies particularly facilitate access to financial services to women, and, unlike marketing cooperatives, also to farm labourers and other poorest-of-the-poor people.
Lessons and hypotheses. A review of the ex post, mid-term review and supervision reports on the projects and grants under study made it possible for IOE to identify a large number of lessons for planning new projects and programmes. The most important of these are summarized under the following headings and subheadings (see also paragraphs 135–149 of the main report):
- Establishment and operations (with the following subheadings: Initiation phase; Joining FOs; Process of formalization; Management problems; and Services of cooperative microfinance institutions);
- Roles of different agencies (Multiple organizations involved; and Decentralization and mainstreaming);
- Coordination and collaboration (Coordination; and Linkages within the sector); and
- Dependency and sustainability (Dependency; and Targeting of support).
As well as learning from individual cases, the study team took a broader view of the project and grant documents and presented a number of hypotheses with regard to topics such as combining multiple beneficiary groups and policies under the same intervention; achieving more rapid start-up of programmes when cooperatives are named as beneficiary groups; dealing with apex organizations' inadequate capacity to lead; assessing the capacity required to operate FOs; and calculating the appropriate length of FO projects and grants (see paragraph 149). As they were based on a relatively small sample, however, these hypotheses will need to be validated.
Validation by "benchmarking." IOE conducted interviews and reviews of documents with other Rome-based United Nations specialised agencies and with World Bank regarding their activities with respect to cooperatives and other FOs. It also reviewed the relevant Internet documents of a number of other multilateral and bilateral agencies as well as those of foundations promoting cooperatives. These reviews showed results similar to the findings of the study on IFAD's engagement with cooperatives. There was a common understanding regarding problems identified by multilateral/bilateral agencies on the one hand, and designers and evaluators of IFAD projects and grants on the other hand, indicating that, while not new, problems relating to cooperative developments differ from case to case.
Similarities also exist with regard to solutions, although tools used by individual agencies may differ. However, although the most suitable types of assistance are well known, it became clear that conditions vary so much across different countries that the general cooperative (or other FO) model needed to be adjusted for each country based on the historical, economic, and social conditions involved. In addition, tools to countermand the problem areas and promote the sustainable development of cooperatives and other FOs must be designed to fit to circumstances. Because there is much accumulated knowledge at the different development institutions, it will not be necessary to seek completely new solutions; instead, close collaboration between the professionals dealing with cooperatives and other FOs in the different development institutions should be promoted, and specialized persons used for situational analyses and programme designs.
The way forward. As noted at the September 2012 workshop, while this study represents an important stage in developing guidelines for dealing with cooperatives and similar organizations, a "way forward" will need to be developed. Recommended follow-up activities include identifying countries with different types of FOs, and conducting a thorough study of their effectiveness and modes of operation; preparing a number of success stories (and potentially viewpoints of the beneficiaries through a few testimonies); expanding the study to cover farmer associations not included in the report; validating the hypotheses presented; and recommending action tailored to different circumstances.
1/ Ethiopia, Morocco, Bosnia-Herzegovina and Guinea.
Democratic Socialist Republic of Sri Lanka: Dry Zone Livelihood Support and Partnership Programme
The Independent Office of Evaluation conducted its first Impact Evaluation in 2013 of an IFAD-funded project: the Sri Lanka Dry Zone Livelihood Support and Partnership Programme. The Impact Evaluation entailed, inter-alia, extensive primary data collection and analysis, including a quantitative survey of over 2,500 households. The survey covered households that benefitted from project support and those that did not (the latter being the control group). The evaluation found that the project played an important role in exposing smallholder farmers to new crops and improved agricultural techniques and can be considered as a pioneering intervention that built momentum, human capital and experience. However the effects on household assets and expenditures were mixed compared to the project’s monitoring and evaluation data and much remains to be done to consolidate the emerging dynamics.
Evaluation synthesis: Results-based Country Strategic Opportunities Programmes
In December 2011, the Executive Board requested the Independent Office of Evaluation of IFAD to prepare a synthesis report on the experience gained in results-based country strategic opportunities programmes (RB-COSOPs) – introduced in 2006. The synthesis sought to evaluate whether RB-COSOPs are serving as instruments for:
- improved country programme planning;
- learning and accountability;
- strengthening synergies between lending and non-lending activities.
COSOPs approved prior to 2006 were largely viewed as internal documents for identifying investment opportunities and pipeline projects, rather than as a tool for strategic, dynamic management of IFAD-supported programmes. They were prepared with limited participation by and input from in-country partners. In contrast, the RB-COSOP emphasizes: alignment with national priorities; joint IFAD and country ownership; synergies between lending and non-lending instruments; results and performance management; learning and accountability; partnership-building and harmonization; and innovations and scaling up. This approach seeks a sharper focus on rural poverty, with more-effective targeting of the poorest people in rural areas. At the time of writing, 50 RB-COSOPs have been approved.
The most recent country programme evaluations (CPEs) have observed that in terms of relevance, RB-COSOPs have generally been aligned with IFAD's and the partner country's strategic objectives. Nevertheless, there are some areas not sufficiently addressed, such as engagement with the private sector, smallholder agriculture, indigenous peoples' issues, and approaches towards scaling up. Overall, RB-COSOP effectiveness, which determines the extent to which the strategic objectives were or are likely to be achieved, is rated well.
CPEs have noted an improvement in overall portfolio performance, which also includes an assessment of rural poverty impact, sustainability, innovation and scaling up. However, it is still too early to assess whether the projects included in RB-COSOPs are implemented significantly better than earlier projects, because they are ongoing and have not yet been evaluated.
Other findings by CPEs indicate scope for improving synergies between different instruments (loans and grants, including other non-lending activities). However, there are a number of limitations that constrain the performance of non-lending activities: insufficient budgetary allocations, if any; limited incentive structure; insufficient in-country human resources; limited grant resources and cumbersome access procedures; and insufficient integration of these activities into overall country programmes.
In general, the importance and utility of the RB-COSOP is widely acknowledged within IFAD as a strategic document that helps guide IFAD's country engagement in line with national priorities, and promotes mutual ownership and accountability. Concerns and uncertainties remain regarding when an RB-COSOP should be undertaken and whether the COSOP cycle (five years) should be aligned with the performance-based allocation system cycle (three years at present). No RB-COSOPs provide an indication of the budget required for their delivery and attainment of their strategic objectives.
With increasing demands for better analytical inputs in COSOP formulation, especially in regard to conducting thorough background studies and analysis of institutional architecture at the country level, concerns remain as to how this can be achieved given the rather meagre resources allocated for COSOP preparation.
While most country programme managers (CPMs) have found the guidelines useful, the review process has proven cumbersome and time-consuming; it has not facilitated an effective dialogue with recipient governments and quite often has not provided significant value added. CPMs are often obliged to spend considerable time responding to the issues raised, sometimes in repeated back-and-forth exchanges. The underlying grievance is that greater importance is given to the views of external reviewers than to those of the CPM accountable for the entire RB-COSOP process. It is felt that it might be necessary to ‘front-load' the review process to avoid diplomatic tangles with the government or contradicting the ownership principle.
A key distinguishing feature of the RB-COSOP is the inclusion of a results management framework (RMF) to track the performance and impact of IFAD's contribution and to promote accountability. While reporting of results and outcomes has improved over the years, significant challenges at project and country programme levels do remain: choosing appropriate indicators that capture outcomes and impacts; linking project goals and outcomes with country-level strategic goals and outcomes; determining realistic goals and expected outcomes; and formulating dedicated composite indicators and targets that measure government performance. Another area of concern continues to be the lack of integration of outputs from IFAD‘s monitoring and evaluation (M&E) systems into national M&E systems.
As in IFAD, all other international financial institutions (IFIs) have evolved their country strategy processes, mainly led by their country offices, with the intention of incorporating the principles of ownership included in the Paris Declaration on Aid Effectiveness, and in particular the principle of measuring results. While IFI country strategies also include an RMF, unique IFAD features are the inclusion of the agreement at completion point whenever a CPE has preceded formulation of the RB-COSOP and of the RMF of the last country strategy.
Compared with IFAD, the most obvious difference lies in the multisectoral dimension of other IFIs. This element adds complexity to the preparation of country strategies. A common feature among all other IFIs is the full involvement of their staff in the entire process, while the use of external consultants is considered an exception. In terms of resources devoted to country strategy preparation, all other IFIs allocate greater resources than IFAD – estimates range from US$250,000 to US$500,000. The contents of strategy documents reflect more similarities than differences. Compared with IFAD COSOPs, there is more attention to the identification of risks and mitigation measures.
Overall, a survey of the evidence available would support the following observations:
- Country programme planning. RB-COSOPs have helped improve effective country programme planning by ensuring better geographical and demographic targeting and alignment with national development goals and the IFAD Strategic Framework 2011-2015. Serious efforts have been made to promote full country ownership of the programmes in accordance with the Paris Declaration. Still, there is work to be done, especially with regard to aid harmonization and adoption of national systems of implementation, monitoring and reporting;
- Synergies between lending and non-lending activities. While RB-COSOPs have led to increased attention to policy dialogue, partnership-building and knowledge management in order to promote innovations and scaling up, the effectiveness of these activities is greatly limited by the fact that they are underresourced. Thus synergies are not adequately created between these activities (including the use of grants) and the desired impacts are not realized;
- Learning and accountability. The RB-COSOP's emphasis on knowledge generation and sharing, as well as on tracking and monitoring of results, has led to a heightened sense of accountability among all key stakeholders and has contributed to improved country programme performance. IFAD is generally seen as a reliable and supportive development partner that contributes a wealth of international experience to this effort. The RB-COSOP is viewed as an important input in strengthening and deepening this relationship, while making IFAD's development contribution more effective; and
- A number of issues might be considered by IFAD Management to further enhance IFAD operations: (i) simplification and streamlining of the RB-COSOP guidelines; (ii) enhanced budgets for RB-COSOP formulation and monitoring; (iii) re-examination of the review process, which is cumbersome and time-consuming, and even ‘front-loading' early in the process; (iv) refocusing of RB-COSOP monitoring, beyond portfolio reviews, on assessing whether projects and non-lending activities together are contributing towards achieving the RB-COSOP's strategic objectives; (v) weakness of M&E systems at project and country programme levels of the RMF hinders the integration of outputs from IFAD's M&E system into the national M&E system and prevents COSOPs from becoming ‘living documents'; and (vi) renaming the RB-COSOP to ‘country partnership strategy' in line with the principles of the most recent declarations on aid effectiveness.
National Microfinance Support Programme
The project performance assessment of the National Microfinance Support Programme was successful in achieving its objectives and meeting major targets with respect to supporting the sector’s ability to deliver financial services to poor women and men, particularly regarding client outreach and loan outstanding. By the end of the project in 2009, improvements were still needed with respect to investment in outreach to the underserved states, emphasis on skill development training for livelihoods promotion, women’s empowerment, a poverty focus and regulations for the sector. The achievements of the project were significantly altered by a crisis in the sector in 2010, which affected microfinance institutions’ portfolio quality and sustainability. This assessment looked at the post-crisis period and noted recent signs of recovery with increased fund flow in the sector and more focus on client protection.
Nepal Country Programme Evaluation
This country programme evaluation covers over a decade of IFAD’s cooperation with Nepal (1999-2012). IFAD’s support during the evaluated period has concentrated on rural poverty alleviation through integrated agricultural and rural development programmes; leasehold forestry; and agricultural value chain development. Overall, the IFAD/Nepal partnership is assessed to be moderately satisfactory, considering improvements in the later part of the period evaluated. The IFAD-supported programme portfolio is rated moderately satisfactory mainly owing to recent improvements in support to leasehold forestry and the satisfactory performance of the IFAD-cofinanced Poverty Alleviation Fund. While the overall portfolio is relevant and many quantitative targets were achieved, sustainability and innovation were less successful, and IFAD-supported programmes had a very wide geographical and thematic spread. Rural finance was the least successful part of the overall portfolio
Rural Finance Sector Programme
The project performance assessment of the Rural Finance Sector Programme in China contributed to increased access to finance by the poor, women and small producers. It also contributed to enhancing the livelihoods of rural households in the project counties by an increase in household assets and production, improvement in food security, and in the strengthening of human and social capital and socio-economic status of women. However, the project originally designed as a sector project to participate in systemic policy reforms could not do so due to a lack of proper institutional arrangements, rapid changing nature of the ongoing reform process and inadequate grasp of the reform issues and institutional complexity of the rural finance sector in China.
Uganda Country Programme Evaluation (2013)
This first country programme evaluation for Uganda covers the cooperation and partnership between IFAD and the Government of Uganda over the period 1998-2011Overall, the portfolio performance is assessed as moderately satisfactory, but with challenges of sustainability and in the provision of rural finance, which is essential for promoting better incomes and food security. IFAD’s main contribution has been in developing the vegetable oil subsector where support has been based on an innovative public-private partnership and a value chain approach. The evaluation underlines that this is indeed a far-reaching achievement. During the first part of the period covered by the evaluation, IFAD contributed considerable staff and financial resources to developing policies and partnerships. In the latter part, it also invested much time in policy dialogue but without being able to effectively persuade the Government to accept its views. With transition to direct supervision and implementation support, which is an important change in IFAD’s operating model leading to better results, less staff time is available for non-lending activities (knowledge management, policy dialogue and partnerships.
ECG paper on Gender equality and development evaluation units
This synthesis was undertaken at the request of the Evaluation Cooperation Group (ECG) members, following a panel discussion of gender evaluations at the 2011 ECG meeting in Washington D.C.
The fi ndings and conclusions expressed herein do not necessarily refl ect the ECG policy nor the views of ECG member institutions.
Yemen Country Programme Evaluation
The country programme evaluation looks at results achieved in the last 17 years (since the last CPE in 1992). Overall these have been positive, with IFAD making a significant contribution towards improving agriculture and rural development in some of the country’s poor, remote and least developed areas, and in gaining a solid reputation for specialized expertise and country experience. The evaluation found that IFAD has made a positive impact towards promoting participatory development and supporting social mobilization in rural areas. In many areas agricultural productivity has improved through enhanced irrigation systems, soil conservation, crop improvements, the diversification of production and the development of small-scale fisheries. Despite the challenges of a conservative society, IFAD has helped to empower women by providing them with economic opportunities and increasing their participation in community decision-making. However, IFAD had only limited success in enhancing poor rural households’ access to financial services, even though some progress has been made in establishing community-based savings and credit groups. Despite severe water shortages, relatively few investments have been made in improving surface water management. The country portfolio has also been affected by capacity constraints in government. Finally, despite the country’s large proportion of children and youth and high youth unemployment, few projects/programmes have focused on youth.
Annual Report on the Results and Impact 2012
This tenth Annual Report on Results and Impact of IFAD Operations (ARRI) presents a review of evaluations completed in 2011, with particular emphasis on highlighting what has changed since the first ARRI in 2003. The overall picture that emerges is positive, but with room for improvement, especially in light of the collective aim to transform the organization from generally moderately satisfactory performance to satisfactory, and if possible, even highly satisfactory results in the near future. On the one hand, there is evidence of clear and significant improvement across most of the criteria assessed by evaluations carried out by the Independent Office of Evaluation of IFAD (IOE). But equally, it is evident that a number of important issues identified in the early ARRIs are yet to be effectively addressed.
Performance and trends
Three broad performance trends can be observed from the 2002-2011 project evaluation data. Ratings for two evaluation criteria – natural resources and the environment, and IFAD’s performance as a partner – have improved over the decade. A second group of evaluation criteria – sustainability, innovation and scaling up, and two rural poverty impact domains (human and social capital and empowerment, and institutions and policies) – show a marked
improvement since 2002-2004, but a more recent decline since a peak in 2006-2008. A final group of evaluation criteria – relevance, effectiveness, efficiency, and the performance of government as a partner – show flat and/or declining performance.
Three other features have not changed over the years. First, a pattern of high project relevance and reasonable effectiveness, but only moderate efficiency and sustainability. Second, the predominance of moderately satisfactory performance. Just 23 per cent of the 2011 ratings were satisfactory or better. And third, the importance of the country context. Performance in fragile states is markedly lower than in non-fragile states.
Eighteen country programme evaluations (CPEs) have been completed since 2006. These show clear improvement in all three non-lending activities since 2006-2008 – knowledge management, partnership building and policy dialogue – but little improvement in the last activity since 2007-2009. Notwithstanding improvement in knowledge management, more efforts are needed at the country level, including a larger allocation of dedicated resources for this purpose. Similarly, as underlined in most past ARRIs, it is time for serious efforts to engage more systematically with the multilateral development banks and selected United
Nations organizations at the country level, including the Rome-based agencies.
Two priorities stand out in respect of country programmes. Most CPEs reveal that IFAD-supported country programmes are largely a collection of individual investment projects. The first priority is thus for better integration of all IFAD-supported activities, which is critical to scaled-up and sustainable impact. The second priority is for a well-resourced country presence, preferably with more immediately outposted country programme managers, especially in large countries with sizeable portfolios.
Forest Resource Management Project
The project facilitated the formation of village resource management committees and producer groups; communities acquired valuable knowledge and skills under the project’s various training programmes in areas such as forestry protection, bee-keeping, rattan and bamboo production, and opportunities were created for women members to manage money and gain greater control over resources and access to knowledge. However, the project benefitted only a fraction of the original target population and scored limited achievements towards increasing the incomes of poor people who depend upon forest resources for their livelihoods. In spite of the efforts at the output level, the project did not achieve most of its objectives, mainly constrained by a complicated legal and institutional context, insufficient focus on value chains and marketing of the products promoted, and weak project management. Key recommendations for future operations focus on ensuring realistic institutional foundations, proven technical and commercial potential of income generating activities and ownership of the project as well as improving monitoring and evaluation systems design and functionality.
Microfinance and Technical Support Project
The project was designed as a unique intervention bundling technical and financial services to support income generating activities for moderately and hard core poor households. In spite of a competitive microfinance market and external challenges outside of the project’s control, the project performed well under the credit component, attracting an impressive number of women participants and leading to good results in food security and agricultural productivity. The training component, judged to be too ambitious and insufficiently coordinated with financial services, performed less well, but some innovations and non-traditional methods in technical support have survived in the post-project period. Recommendations of this assessment focus on effectiveness of inputs. The quality of women’s participation would also improve with customized services and products. Other recommendations look at how to better integrate technical support and financial services, and improve the selection of financial institutions as partners to ensure sustainability.
Rural Business Development Programme
In the process of facilitating post-independence economic transition in Moldova, IFAD provided relevant and timely interventions in identified bottleneck areas of rural business development. The project performance assessment of the Rural Business Development Programme in Moldova increased the access of rural entrepreneurs to finance, facilitated business growth of enterprises along selected commodity chains and built small-scale infrastructure for rural businesses. The most important lesson learned from the programme is related to the challenges in ensuring the poverty reduction effect of rural enterprise development. The programme did not include poverty reduction goals, nor did it explicitly target poor households in economic activities or monitor the spill-over effect of rural enterprise development in benefitting or possibly affecting the rural poor. Income changes of households below the poverty line were not sufficiently monitored, which undermined the programme’s achievements.
Northern Mindanao Community Initiatives and Resource Management Project
The IFAD-supported Northern Mindanao Community Initiatives and Resource Management Project achieved positive results, having overcome a slow start-up and initial difficulties with the flow of funds. The project met and in some instances surpassed - its output targets towards establishing and strengthening community institutions and self-help groups, participatory planning, infrastructure improvements, support to indigenous populations, and policy and institutional development. At the same time, impact on agriculture has been moderate and mainly linked to improved irrigation. The number of people experiencing food shortages has declined and the income generated has enabled them to purchase food. However, household food security is still vulnerable to climatic shocks with communities lacking opportunities to move to larger-scale initiatives. Among the key recommendations of this assessment are the need to continue nurturing the strong links between communities and local government units, thus creating potential for further development initiatives that could reduce such vulnerability; and developing a new set of skills to support commercial (value chain) development activities.
Community-Based Rural Development Project in Kampong Thom and Kampot
The objectives of the Community-Based Rural Development Project in Kampong Thom and Kampot in Cambodia were to increase food production and farm income and increase capacity of the poor to use the services available from the Government and other sources for their social and economic development. According to the present assessment, the project’s support to farmers contributed to increased agricultural and livestock production and better livelihoods through improved infrastructure and related access to services and markets. The project also made use of famer-to-farmer extension methods which have spread beyond the immediate demonstration areas. It was a front-runner in terms of putting Cambodia’s decentralization policy into practice.
There were some weaknesses in the hydrological and engineering design and the site selection, which impacted on the functionality of irrigation systems. Operation and maintenance of irrigation but also of roads remain a challenge, although the sustainability of the extension services is likely to benefit from the emerging commercialization, market linkages and resultant contract farming. The assessment recommends that future IFAD-supported projects pay more attention to the role of markets and facilitate farmer promoters to become agricultural input suppliers, in order to strengthen the rural development process.
Rural Recovery and Development Programme
Burundi’s major challenge, at the time of start-up of the Rural Recovery and Development Programme, was to reactivate the economy in the wake of the crisis by restoring the production base and addressing the structural problems affecting production. The programme response was to improve food security and living standards, increase rural incomes and improve land conservation in four of the country’s poorest provinces.
The programme successfully achieved its objectives vis-à-vis the main target populations, in particular the most vulnerable households which had lost their means of productions. It had strong influence on government policies and institutions, with its greatest success lying in its innovations and their scaling up throughout the country. Its interventions also show good signs of sustainability, specifically the high degree of ownership by local populations and good financial returns on several activities.
The assessment notes some weaknesses in the conservation of agricultural land and environmental resources and remaining challenges to ensure long-term sustainability, particularly in consolidating infrastructure management committees.