Non-lending Activities in the Context of South-South Cooperation: Evaluation Synthesis
Support to South-South cooperation (SSC) has been a high priority for IFAD and its Member States since the consultation process of the Eighth Replenishment of IFAD's Resources.
The Independent Office of Evaluation of IFAD (IOE) conducted a review and analysis of IFAD's support to South-South cooperation between 2009 and 2015. SSC support by IFAD has mainly taken the form of knowledge-sharing, with the more programmatic initiatives often financed through grants.
The majority of these initiatives have supported mutual and horizontal SSC embedded in regional and subregional processes with regional/global grants, and occasionally country grants to support emerging economies in capturing and sharing knowledge. One of IFAD's advantages clearly lies in its focus on reducing poverty by investing in rural people, and its accumulated on-the-ground experience. At the same time, results orientation tends to be weak, with outputs often being the main focus of planning and reporting on activities.
There is also diverse understanding at IFAD and among Member States about what South-South cooperation is and implies for IFAD. There are opportunities for IFAD to support South-South cooperation in a more strategic, innovative and effective manner.
FAO's and IFAD's Engagement in Pastoral Development
FAO's and IFAD's Engagement in Pastoral Development Joint Evaluation Synthesis.
The Food and Agriculture Organization of the United Nations (FAO) and the International Fund for Agricultural Development (IFAD) have a history of engagement in pastoral development which is likely to continue. In the face of growing challenges posed by climate change, as well as new economic and political realities, pastoralism offers a production system that prospers in landscapes where other livelihood systems are either at their limit or require large investments. In the 2003-2013 decade - the period reviewed by the joint evaluation synthesis - a total of 194 projects related to pastoral development were formulated and US$1.2 billion were allocated. Areas of intervention included animal health and veterinary services, capacity-building, emergency response, rangeland management and policy dialogue. The evaluation finds that FAO's and IFAD's engagement in pastoral development has been significant, but lacks a coherent conceptual framework and systematic direction. There have been positive results in projects seeking to reduce poverty and hunger by introducing innovative solutions in community-based animal health and natural resource management. However, the evaluation finds that there is still considerable confusion between pastoral development and livestock development, and no clear understanding of pastoral systems, including the specificity of pastoral poverty. This has led to a focus on sedentary activities, and a considerable degree of hit-and miss in the results. The recommendations focus on the need for FAO and IFAD to create policies of engagement in pastoral development and build capacity for systemic engagement in pastoral systems through, for example, developing a better understanding of pastoral systems and their relation to dryland economies. Furthermore, risk-management and resilience strategies should be prepared. They need to be highly context-specific and include a distinction between risk management and risk reduction. The evaluation also highlights the need for the two agencies to strengthen advocacy by pastoralists and on behalf of pastoralists.
IFAD's Engagement with Indigenous Peoples: Evaluation Synthesis
IFAD has been financing projects in support of indigenous peoples since 1979, in particular in Latin America and Asia. According to United Nations estimates, there are more than 370 million indigenous people worldwide spread across some 70 countries. They continue to be overrepresented among the poor: while they constitute just 5 per cent of the world's population, they account for 15 per cent of the world’s poor people.
This evaluation synthesis report confirms that IFAD is in a unique position among development agencies to support indigenous peoples' social and economic empowerment. Given its mandate and focus – rural poverty reduction with attention to the vulnerable and marginalized, participatory approaches, community development, empowerment and social inclusion, IFAD has naturally followed a proactive approach to supporting indigenous peoples.
In light of its unique position and comparative advantage, and building on its experiences so far, there is still room to strengthen the consistent implementation of IFAD's policy on engagement with indigenous peoples, in particular at the level of investment projects.
The report highlighted the importance of paying greater attention to key project design elements, such as devising tailored and differentiated approaches to build on the culture, identity and knowledge of the indigenous peoples' communities.
The report also highlighted the importance of enhancing staff understanding on indigenous peoples' issues. IFAD could further strengthen knowledge management in this area, taking advantage of substantial experience, lessons and knowledge of engagement with indigenous peoples.
Rural Youth Evaluation synthesis report
Evaluation Synthesis. IFAD started enhancing its focus on rural youth in the last decade and particularly in 2010, when the Strategic Framework for the period 2011–2015 reflected the Fund's attention and commitment in promoting rural youth development. Overall, among the projects selected for this evaluation, about 83 per cent of past pro-youth projects targeted rural youth explicitly and 42 per cent included youth-specific activities. The evaluation also highlights that IFAD is equipped with ad-hoc strategies, policies and guidelines to work with rural youth. Looking forward, the evaluation offers a number of reflections for IFAD, to ensure that rural youth plays a catalyst role in rural transformation and agricultural sustainability: mainstreaming youth across country programmes in all regions; investing in the update of the knowledge base on youth and adequate socio-economic profiling; resolving the issue of efficiency versus equity upfront at design stage in terms of target group identification; adopting systematically age-disaggregated monitoring indicators, to foster IFAD's learning and replication processes; and enhancing strategic partnerships to support the scaling up of successful and innovative models.
IFAD’s Engagement in Middle-income Countries: Evaluation Synthesis
The evaluation synthesis concludes that there is a solid cause for IFAD’s continued engagement in middle-income countries (MICs). In fact, IFAD remains a relevant and highly valued partner for MICs, where there is an extensive demand for the Fund’s assistance, given its specialization and comparative advantage in working in remote rural areas and inclusive growth. However, taking into account the heterogeneity of these countries, there are opportunities for IFAD to further sharpen some of its existing products and instruments, devoting greater attention to non-lending activities, technical assistance and South-South and triangular cooperation. In addition, IFAD should intensify its ongoing efforts to mobilize additional funding and to strengthen strategic partnerships with other bilateral and multilateral development organizations.
Water Conservation and Management Evaluation synthesis report
Evaluation synthesis. Overall, IFAD’s engagement in the water sector has improved also as a result of better performance in related sectors such as financial services, value addition and market development. Despite not having a specific policy on water, as the other multilateral development banks, IFAD compares well in regard to agricultural water management, in which it has a distinct comparative advantage. The scenario emerging on the water front presents a set of challenges and opportunities such as: water is both a constraint to development and an opportunity for innovation; water productivity is critical to enhancing development effectiveness; and rainfed farming is now key to increasing food production and agricultural productivity. This is where IFAD, in partnership with local governments and other development agencies, can take the lead in developing a strategy that can bring about a “brown revolution” in rainfed agriculture akin to the “green revolution” of irrigated agriculture.
Rural differentiation and smallholder development
This synthesis report is based on two main sources of information:
- review of current external literature; and
- IFAD sources (a review of IOE evaluations, Programme Management Department (PMD) documentation on targeting, and IFAD policy and strategy documents.
Section II draws examples from the development literature of analyses of differences among rural populations, and potential pathways for the development of various rural groups. It then relates these to a consideration of the policies that might facilitate the progress of groups along these paths.
Section III focuses on IFAD’s experience in targeting.
The final section summarizes the key issues and IFAD’s options for strengthening its impact on poverty and hunger through approaches that address rural diversity and maximize the pro-poor benefits of rural and smallholder agricultural development.
The annex contains a more in-depth analysis of the potential of different types of rural and agricultural development policies to benefit diverse categories of rural people, and IFAD’s potential role in relation to those policies.
IFAD’s Engagement with Cooperatives - A Study in Relation to the United Nations International Year of Cooperatives
Cooperatives are not the only farmers' organizations (FOs) to receive IFAD support. However, because the United Nations has designated 2012 as the International Year of Cooperatives (IYC), it was considered appropriate that the synthesis report should focus mainly on cooperatives. Other forms of FOs are also discussed, not only owing to their importance but also because IFAD documents often do not distinguish between cooperatives and other types of FO.
In the context of the IYC, the Executive Board requested the Independent Office of Evaluation of IFAD (IOE) to study the Fund's engagement with cooperatives, particularly with regard to IFAD's evolving approaches to supporting cooperatives, lessons learned and good practices thatwould contribute to furtherdebate and reflection on the topic. To that end, IOE reviewed IFAD's policy and strategy documents relating to a limited but select sample of projects and grants, which revealed that IFAD has provided a wide range of financial support to cooperatives and similar organizations. A typology was also drawn up of planned and actual operations and lessons learned in designing and implementing projects and grants relating to cooperatives and similar FOs. However, to determine the full extent of IFAD's financial support to such organizations would have called for a much larger sample than the allocated resources allowed for the task.
IFAD's policies and approaches. IFAD has no stand-alone policy on cooperatives or other FOs, and the term "cooperatives" is not specifically mentioned in its strategy and other official documents. For at least the last two decades, IFAD has consistently promoted organizations with values and principles similar to those of cooperatives, and, when discussing support provided above the farm level, its annual reports or policy documents have used more general terms. From the late 1990s, and especially since the year 2000, the terms "farmers' organizations" and "rural people's organizations" appearing in the Fund's annual reports have implicitly encompassed agricultural cooperatives. In the IFAD Rural Finance Policy of 2000 (updated in 2009), the term "rural financial institutions" also encompasses cooperative savings and credit associations.
Terminology and study material. Because IFAD has not distinguished cooperatives from other FOs, and as most FOs are membership-based and democratically led, it was considered appropriate that they should be included in the study. The definition of FOs recommended by the IFAD-sponsored Farmers' Forum has been used, that is: "Farmer organizations are membership-based organizations of smallholders, family farmers and rural producers – including pastoralists, artisanal fishers, landless people and indigenous people – that are structured beyond the grass-roots or community levels, to local, national, regional and global levels." Cooperatives, which are also membership-based and democratically managed organizations, fall under this definition, as do non-cooperative, registered farmers' associations and economic farmers' associations. Local farmer groups and village development committees have not been included. Altogether, 25 projects from the five IFAD regional divisions and 10 grants, usually supporting multi-country activities, were selected for the study inasmuch as they included components in support of FOs.
Typology of IFAD's assistance to cooperatives and similar FOs. IOE reviewed the material (project and grant design, appraisal and President's Reports and Recommendations, plus mid-term review, supervision, ex post evaluation and project status reports) for information on the IFAD assistance that was planned and results actually achieved. Items of special interest were activities considered important for the development of cooperatives and other FOs, and for IFAD, and involved the inclusion of FOs in the goals and objectives of projects; components in support of such organizations; problems (challenges) identified and assistance planned and executed; governance issues; beneficiaries; gender promotion; and risk analyses regarding cooperatives and other FOs. The results of the findings are given in the main report but, for the purpose of this summary, the following aspects are highlighted.
Although establishing cooperatives and other FOs, and building up their networks and vertical support organizations, are demanding tasks, their management – especially of cooperatives – is even more demanding, particularly owing to their democratic nature and large, often poorly educated membership. The project and grant designers identified many potential problems, which the study team classified into the following broad groups:
- Inadequate organization by rural populations;
- Poor reputation of cooperatives;
- Effects of economic liberalization;
- Lack of experience and relative financial weakness of cooperatives;
- Lack of competencies and systems;
- Poor infrastructure;
- Lack of vertical integration and linkages; and
- Inadequate finance.
The measures used to deal with such problems included:
- Building up capacity (usually in the form of technical assistance, training, workshops and study visits) at all levels: for staff, committee members and ordinary members of cooperatives and other FOs;
- Strengthening institutions (helping to establish FOs and providing technical assistance or funds for basic infrastructure, equipment, and technical or managerial backstopping);
- Provision and strengthening of financial services;
- Technical assistance for special tasks and studies; and
- Supporting vertical structures for advocacy and policy dialogue with government and traders.
These measures, applied selectively in individual projects and grants, generally produced at least moderately satisfactory results. In view of resource limitations, IOE was unable to evaluate the individual projects (and, in any event, only five projects had completed post-evaluations reports). However, the mid-term and supervision reports showed that all but four1 had average ratings of 4 or more (from "moderately satisfactory" to "satisfactory"); that projects rated below 4 were, on average, deemed as above "moderately unsatisfactory"; and that, with regard to institution-building, even the four weaker projects were rated above 4. In IOE's view, these ratings would have been higher had situational (economic, social, educational, historical and even psychological situations) and institutional (needs for capacity-building and institutional strengthening) analyses been undertaken before project/programme design.
IFAD has used cooperative projects – and to some extent other FO projects – to reach large numbers of beneficiaries and promote women's participation. In many places as much as 30 per cent or more of the thousands of beneficiaries are women. Cooperative savings and credit societies particularly facilitate access to financial services to women, and, unlike marketing cooperatives, also to farm labourers and other poorest-of-the-poor people.
Lessons and hypotheses. A review of the ex post, mid-term review and supervision reports on the projects and grants under study made it possible for IOE to identify a large number of lessons for planning new projects and programmes. The most important of these are summarized under the following headings and subheadings (see also paragraphs 135–149 of the main report):
- Establishment and operations (with the following subheadings: Initiation phase; Joining FOs; Process of formalization; Management problems; and Services of cooperative microfinance institutions);
- Roles of different agencies (Multiple organizations involved; and Decentralization and mainstreaming);
- Coordination and collaboration (Coordination; and Linkages within the sector); and
- Dependency and sustainability (Dependency; and Targeting of support).
As well as learning from individual cases, the study team took a broader view of the project and grant documents and presented a number of hypotheses with regard to topics such as combining multiple beneficiary groups and policies under the same intervention; achieving more rapid start-up of programmes when cooperatives are named as beneficiary groups; dealing with apex organizations' inadequate capacity to lead; assessing the capacity required to operate FOs; and calculating the appropriate length of FO projects and grants (see paragraph 149). As they were based on a relatively small sample, however, these hypotheses will need to be validated.
Validation by "benchmarking." IOE conducted interviews and reviews of documents with other Rome-based United Nations specialised agencies and with World Bank regarding their activities with respect to cooperatives and other FOs. It also reviewed the relevant Internet documents of a number of other multilateral and bilateral agencies as well as those of foundations promoting cooperatives. These reviews showed results similar to the findings of the study on IFAD's engagement with cooperatives. There was a common understanding regarding problems identified by multilateral/bilateral agencies on the one hand, and designers and evaluators of IFAD projects and grants on the other hand, indicating that, while not new, problems relating to cooperative developments differ from case to case.
Similarities also exist with regard to solutions, although tools used by individual agencies may differ. However, although the most suitable types of assistance are well known, it became clear that conditions vary so much across different countries that the general cooperative (or other FO) model needed to be adjusted for each country based on the historical, economic, and social conditions involved. In addition, tools to countermand the problem areas and promote the sustainable development of cooperatives and other FOs must be designed to fit to circumstances. Because there is much accumulated knowledge at the different development institutions, it will not be necessary to seek completely new solutions; instead, close collaboration between the professionals dealing with cooperatives and other FOs in the different development institutions should be promoted, and specialized persons used for situational analyses and programme designs.
The way forward. As noted at the September 2012 workshop, while this study represents an important stage in developing guidelines for dealing with cooperatives and similar organizations, a "way forward" will need to be developed. Recommended follow-up activities include identifying countries with different types of FOs, and conducting a thorough study of their effectiveness and modes of operation; preparing a number of success stories (and potentially viewpoints of the beneficiaries through a few testimonies); expanding the study to cover farmer associations not included in the report; validating the hypotheses presented; and recommending action tailored to different circumstances.
1/ Ethiopia, Morocco, Bosnia-Herzegovina and Guinea.
Evaluation synthesis: Results-based Country Strategic Opportunities Programmes
In December 2011, the Executive Board requested the Independent Office of Evaluation of IFAD to prepare a synthesis report on the experience gained in results-based country strategic opportunities programmes (RB-COSOPs) – introduced in 2006. The synthesis sought to evaluate whether RB-COSOPs are serving as instruments for:
- improved country programme planning;
- learning and accountability;
- strengthening synergies between lending and non-lending activities.
COSOPs approved prior to 2006 were largely viewed as internal documents for identifying investment opportunities and pipeline projects, rather than as a tool for strategic, dynamic management of IFAD-supported programmes. They were prepared with limited participation by and input from in-country partners. In contrast, the RB-COSOP emphasizes: alignment with national priorities; joint IFAD and country ownership; synergies between lending and non-lending instruments; results and performance management; learning and accountability; partnership-building and harmonization; and innovations and scaling up. This approach seeks a sharper focus on rural poverty, with more-effective targeting of the poorest people in rural areas. At the time of writing, 50 RB-COSOPs have been approved.
The most recent country programme evaluations (CPEs) have observed that in terms of relevance, RB-COSOPs have generally been aligned with IFAD's and the partner country's strategic objectives. Nevertheless, there are some areas not sufficiently addressed, such as engagement with the private sector, smallholder agriculture, indigenous peoples' issues, and approaches towards scaling up. Overall, RB-COSOP effectiveness, which determines the extent to which the strategic objectives were or are likely to be achieved, is rated well.
CPEs have noted an improvement in overall portfolio performance, which also includes an assessment of rural poverty impact, sustainability, innovation and scaling up. However, it is still too early to assess whether the projects included in RB-COSOPs are implemented significantly better than earlier projects, because they are ongoing and have not yet been evaluated.
Other findings by CPEs indicate scope for improving synergies between different instruments (loans and grants, including other non-lending activities). However, there are a number of limitations that constrain the performance of non-lending activities: insufficient budgetary allocations, if any; limited incentive structure; insufficient in-country human resources; limited grant resources and cumbersome access procedures; and insufficient integration of these activities into overall country programmes.
In general, the importance and utility of the RB-COSOP is widely acknowledged within IFAD as a strategic document that helps guide IFAD's country engagement in line with national priorities, and promotes mutual ownership and accountability. Concerns and uncertainties remain regarding when an RB-COSOP should be undertaken and whether the COSOP cycle (five years) should be aligned with the performance-based allocation system cycle (three years at present). No RB-COSOPs provide an indication of the budget required for their delivery and attainment of their strategic objectives.
With increasing demands for better analytical inputs in COSOP formulation, especially in regard to conducting thorough background studies and analysis of institutional architecture at the country level, concerns remain as to how this can be achieved given the rather meagre resources allocated for COSOP preparation.
While most country programme managers (CPMs) have found the guidelines useful, the review process has proven cumbersome and time-consuming; it has not facilitated an effective dialogue with recipient governments and quite often has not provided significant value added. CPMs are often obliged to spend considerable time responding to the issues raised, sometimes in repeated back-and-forth exchanges. The underlying grievance is that greater importance is given to the views of external reviewers than to those of the CPM accountable for the entire RB-COSOP process. It is felt that it might be necessary to ‘front-load' the review process to avoid diplomatic tangles with the government or contradicting the ownership principle.
A key distinguishing feature of the RB-COSOP is the inclusion of a results management framework (RMF) to track the performance and impact of IFAD's contribution and to promote accountability. While reporting of results and outcomes has improved over the years, significant challenges at project and country programme levels do remain: choosing appropriate indicators that capture outcomes and impacts; linking project goals and outcomes with country-level strategic goals and outcomes; determining realistic goals and expected outcomes; and formulating dedicated composite indicators and targets that measure government performance. Another area of concern continues to be the lack of integration of outputs from IFAD‘s monitoring and evaluation (M&E) systems into national M&E systems.
As in IFAD, all other international financial institutions (IFIs) have evolved their country strategy processes, mainly led by their country offices, with the intention of incorporating the principles of ownership included in the Paris Declaration on Aid Effectiveness, and in particular the principle of measuring results. While IFI country strategies also include an RMF, unique IFAD features are the inclusion of the agreement at completion point whenever a CPE has preceded formulation of the RB-COSOP and of the RMF of the last country strategy.
Compared with IFAD, the most obvious difference lies in the multisectoral dimension of other IFIs. This element adds complexity to the preparation of country strategies. A common feature among all other IFIs is the full involvement of their staff in the entire process, while the use of external consultants is considered an exception. In terms of resources devoted to country strategy preparation, all other IFIs allocate greater resources than IFAD – estimates range from US$250,000 to US$500,000. The contents of strategy documents reflect more similarities than differences. Compared with IFAD COSOPs, there is more attention to the identification of risks and mitigation measures.
Overall, a survey of the evidence available would support the following observations:
- Country programme planning. RB-COSOPs have helped improve effective country programme planning by ensuring better geographical and demographic targeting and alignment with national development goals and the IFAD Strategic Framework 2011-2015. Serious efforts have been made to promote full country ownership of the programmes in accordance with the Paris Declaration. Still, there is work to be done, especially with regard to aid harmonization and adoption of national systems of implementation, monitoring and reporting;
- Synergies between lending and non-lending activities. While RB-COSOPs have led to increased attention to policy dialogue, partnership-building and knowledge management in order to promote innovations and scaling up, the effectiveness of these activities is greatly limited by the fact that they are underresourced. Thus synergies are not adequately created between these activities (including the use of grants) and the desired impacts are not realized;
- Learning and accountability. The RB-COSOP's emphasis on knowledge generation and sharing, as well as on tracking and monitoring of results, has led to a heightened sense of accountability among all key stakeholders and has contributed to improved country programme performance. IFAD is generally seen as a reliable and supportive development partner that contributes a wealth of international experience to this effort. The RB-COSOP is viewed as an important input in strengthening and deepening this relationship, while making IFAD's development contribution more effective; and
- A number of issues might be considered by IFAD Management to further enhance IFAD operations: (i) simplification and streamlining of the RB-COSOP guidelines; (ii) enhanced budgets for RB-COSOP formulation and monitoring; (iii) re-examination of the review process, which is cumbersome and time-consuming, and even ‘front-loading' early in the process; (iv) refocusing of RB-COSOP monitoring, beyond portfolio reviews, on assessing whether projects and non-lending activities together are contributing towards achieving the RB-COSOP's strategic objectives; (v) weakness of M&E systems at project and country programme levels of the RMF hinders the integration of outputs from IFAD's M&E system into the national M&E system and prevents COSOPs from becoming ‘living documents'; and (vi) renaming the RB-COSOP to ‘country partnership strategy' in line with the principles of the most recent declarations on aid effectiveness.
ECG paper on Gender equality and development evaluation units
This synthesis was undertaken at the request of the Evaluation Cooperation Group (ECG) members, following a panel discussion of gender evaluations at the 2011 ECG meeting in Washington D.C.
The fi ndings and conclusions expressed herein do not necessarily refl ect the ECG policy nor the views of ECG member institutions.