Ethiopia Country Programme Evaluation
Country Programme Evaluation
This is the second country programme evaluation conducted by Independent Office of Evaluation in Ethiopia and it covers eight years of IFAD's operations (2007-2015) in the country. Since 1980, IFAD has provided US$473 million to finance 17 projects with a total cost of more than US$1.2 billion. Over the last few years, IFAD has been one of Ethiopia's major development partners in areas such as small-scale irrigation, rural finance, natural resources management and support to the development of pastoral communities. Overall, portfolio performance has been satisfactory, with good achievements in human and social capital and good alignment with the Government's decentralization policy. Sustainability, scaling up and gender were also satisfactory. Areas that need to be improve include policy dialogue and impact on institutions and policies. Weak monitoring and evaluation hinders evidence-based policy engagement. Finally, the CPE found that the country programme spreads too thinly over many thematic areas, and recommended that IFAD should focus on thematic areas where has a comparative advantage, such as small-scale irrigation, rural finance and pastoral community development. The CPE also recommends using a longer-term programmatic approach to lending, based on good experiences in the past, and greater focus on non-lending services.
Environment and Natural Resource Management: Evaluation Synthesis
The Independent Office of Evaluation of IFAD has reviewed the Fund's support of Environment and Natural Resource Management (ENRM) through an examination of 72 evaluations conducted between 2010 and 2015. IFAD has taken a number of steps to strengthen its integration of ENRM issues into its operations - relating to both avoiding harm and doing good: the Environment and Climate Division has been established, the environmental and social safeguards have been upgraded and the Adaptation for Smallholder Agriculture Programme has been launched. From the analysis of the evaluation reports, it is clear that these measures are responding to evidence from the projects in the Evaluation Synthesis sample. However, the analysis shows that alignment with ENRM policies in IFAD country strategies is mixed during the period covered. A small number of country strategic opportunities programmes show a clear progression to a stronger focus on ENRM; others reveal a shift in the direction to other priority strategic areas, such as value chain investments, without necessarily paying attention to ENRM. Moreover, spending on ENRM, measured by conventional sub-component categories excluding ASAP, has not increased greatly as a proportion of IFAD's overall budget during the period 2005-2015. Although data on ENRM content in loans are incomplete and probably understate the actual amount, they suggest some challenges in integrating environmental stewardship effectively into country programmes.
Non-lending Activities in the Context of South-South Cooperation: Evaluation Synthesis
Support to South-South cooperation (SSC) has been a high priority for IFAD and its Member States since the consultation process of the Eighth Replenishment of IFAD's Resources.
The Independent Office of Evaluation of IFAD (IOE) conducted a review and analysis of IFAD's support to South-South cooperation between 2009 and 2015. SSC support by IFAD has mainly taken the form of knowledge-sharing, with the more programmatic initiatives often financed through grants.
The majority of these initiatives have supported mutual and horizontal SSC embedded in regional and subregional processes with regional/global grants, and occasionally country grants to support emerging economies in capturing and sharing knowledge. One of IFAD's advantages clearly lies in its focus on reducing poverty by investing in rural people, and its accumulated on-the-ground experience. At the same time, results orientation tends to be weak, with outputs often being the main focus of planning and reporting on activities.
There is also diverse understanding at IFAD and among Member States about what South-South cooperation is and implies for IFAD. There are opportunities for IFAD to support South-South cooperation in a more strategic, innovative and effective manner.
Rural Diversification Programme
The goal of IFAD's Rural Diversification Programme in Mauritius was to (i) broaden the income and resource base of the target population of small and marginal planters, fishermen, unemployed, landless and female-headed households and (ii) improve their technical and entrepreneurial capacity through training and the strengthening of grass-roots groups and organizations.
While the programme was found to have contributed to increased income levels and food security through higher agricultural production and fish catches, and some community projects have supported poor women to overcome social and economic exclusion, the overall achievement of the development objectives was deemed moderately unsatisfactory. The programme came short of meeting its objectives of decreased farmer dependency on sugar cane, the environmental impact remained high, and there was a failure to address critical issues such as human resources constraints and not internalizing previous learning due to weak monitoring and learning. For future projects, the report recommends more attention being paid to support value chains and market linkages, improved access to credit and methods to increase community ownership.
Oasis Sustainable Development Programme
The overall goal of the Oasis Sustainable Development Programme was to promote oasis development by enabling the empowerment of oasis communities to participate effectively in the pursuit of the national objectives for poverty reduction and the fight against environmental degradation.
The programme's stronger points are related to its relevance and alignment with the priorities of the Government and IFAD and with the needs of the target group living in a rural environment, subject to high levels of climate risk, erosion, natural resource degradation, water shortages, poverty and emigration.
Its successes have laid the basis to generate the desired socio-economic transformations in the oasis environment. Notwithstanding these globally positive achievements in terms of relevance, effectiveness and rural poverty impact, major challenges remain:
- the low likelihood of sustainability of infrastructures that may jeopardize the long-term durability of the project's achievements;
- a modest level of policy dialogue which has not enabled the base institutions to integrate their work into communal institutional planning;
- and the capacities of water user associations in place, which remains fragile.
Turkey Country Programme Evaluation
Over the last 30 years, IFAD has supported 10 projects in Turkey for a total of cost of US$661.1 million, of which IFAD has provided $189 million. Activities have directly benefited 1.3 million households. The geographical focus has been on the poorer and disadvantaged regions and provinces of the country. The evaluation highlights that the long-standing partnership between IFAD and the government of Turkey is strategically important to both. Beyond IFAD's financing role, there has been demand for IFAD to be a more active player in sharing its technical expertise and international knowledge and experience, as a way to enrich the partnership, especially in implementing the 2030 Agenda for Sustainable Development in Turkey. The IFAD-funded projects have helped improve the incomes and quality of life of the rural poor through development of rural infrastructure; made advances in increasing agricultural productivity and supporting commercialization; and achieved modest gains in rural employment and self-sustaining institutions of the rural poor. However, despite continued government support, the evaluation found that the scope for sustainability of interventions was limited by weak operation and maintenance arrangements in infrastructure and insufficient collaboration with the rural financial sector.
IFAD’s Performance-based Allocation System
IFAD’s Performance-based Allocation System
Since it was introduced by IFAD's Executive Board in 2003, the Performance-based Allocation System (PBAS) has enhanced the Fund's credibility, transparency and predictability of financial resource allocations to its developing Member States. The core feature of IFAD's PBAS is that country allocations are calculated using a specific formula to generate a country score, using several variables that, put together, determine country needs and country performance. Overall, the PBAS is found to be relevant. The formula should better factor in some key dimensions of IFAD's priorities, such as food security, nutrition and climate change. It also should improve the way it considers vulnerability issues as determinants of country needs. The evaluation finds the system's effectiveness to be on the whole moderately satisfactory. The rationale for including or excluding countries from the PBAS and the underlying mechanisms guiding the capping system should be made more explicit and institutionalized. Among the recommendations, the need to refine the PBAS design, by sharpening its objectives and strengthening the rural poverty focus; streamline the process for better effectiveness; and enhance management and governance, by taking a more corporate approach to the PBAS in general.
|In-house learning event on the CLE-PBAS, 9 March 2016, IFAD headquarters. Photo by Maurizio Navarra.|
Pastoral Community Development Project II
The Pastoral Community Development Project in Ethiopia successfully introduced community-driven development, which was key in empowering pastoralist communities. Overall, the project performed strongly on its major components, which included improving livelihoods through access to social and economic infrastructure and financial services. Moreover, good progress was made towards gender and women’s empowerment. The effects of better access of pastoral women to education and health, and their economic empowerment, could contribute to profound changes in their communities.
There are, however, some areas which need to be addressed and improved during the third phase currently under way. These concern the inclusion of local knowledge and social and environmental effects due to climate change and the sustainability of benefits generated. In addition, the assessment recommends that the project ensures the mobility of pastoralists is respected, and that the project engages in a more open dialogue with other Ethiopian ministries and actors, including development partners active in pastoral development.
Environment Conservation and Poverty-Reduction Programme in Ningxia and Shanxi
The Environment Conservation and Poverty-Reduction Programme in Ningxia and Shanxi aimed at sustainable and equitable poverty reduction for 300,000 vulnerable rural households living in an environment with a fragile and deteriorating natural resource base. A large number of these impoverished households belonged to the Hui ethnic minority. The programme used a multisectoral approach to address the main causes of poverty within a fragile environment. Programme activities were implemented in the context of wider Government efforts to reduce poverty in the region. The scale of the concerted efforts enabled a far-reaching transformation of the rural landscape, with significant improvements in basic education and health services, and consolidation of the natural resource base. Value was added through the consistent efforts to target programme benefits to the poorer households, in particular women. Although it has delivered a number of results, the programme failed to stand up to its original intention and purpose. In particular it did not deliver the transformative approaches or innovative practices that could have informed ongoing Government programmes and policies for poverty reduction in environmentally sensitive areas. Analysis of the factors limiting the relevance, effectiveness and impact of this programme highlights the need for IFAD to keep abreast of China's rapid development.
Microfinance and Microenterprise Development Project
The Microfinance and Microenterprise Development Project in Djibouti aimed to establish a viable network of savings and credit unions; promote microenterprises services and develop sustainable non-financial services; formulate and adopt a national microfinance and microenterprise strategy with a legal framework; and strengthen and diversify income-generating activities. Overall, the project 's contribution is important in terms of reform of the microfinance sector, particularly through the adoption of legal texts and strategies, and partnerships with institutions specialized in microfinance. The project has advocated for an inclusive approach that has promoted women's participation in decision-making regarding the governance of savings and credit unions. However, sustainability of benefits has been hampered by human resources constraints of savings institutions and credit unions, the absence of an apex structure, the continued importance of subsidies, weak profitability of the portfolio and weak savings and credit unions operational and financial autonomy. The evaluation recommended that future interventions focus on strengthening the network of credit and savings banks so that they can better serve the target group.
Agricultural Investments and Services Project
The most relevant achievement of the Agricultural Investments and Services Project in the Kyrgyz Republic was the advance made with the pasture reform. In particular, the project contributed to improved and more equitable access to pastures by livestock farmers and herders, based on a combination of an enabling legislative framework (following the passing of the Pasture Law), broad-based inclusive social mobilization, local capacity-building activities and support to pasture infrastructure. While the achievements to date serve as a good basis for sustainability, it is important to continue with awareness-building and capacity-building at community level to empower them to nurture and enhance the pasture quality more proactively. The project also made significant contribution to improving veterinary service delivery and reduction in animal and zoonotic disease incidence. However, the country's veterinary services still lack necessary capacity and mandate clarity. Without more decisive commitment and sufficient budget allocation by the Government, benefits generated by the project could be lost.
FAO's and IFAD's Engagement in Pastoral Development
FAO's and IFAD's Engagement in Pastoral Development Joint Evaluation Synthesis.
The Food and Agriculture Organization of the United Nations (FAO) and the International Fund for Agricultural Development (IFAD) have a history of engagement in pastoral development which is likely to continue. In the face of growing challenges posed by climate change, as well as new economic and political realities, pastoralism offers a production system that prospers in landscapes where other livelihood systems are either at their limit or require large investments. In the 2003-2013 decade - the period reviewed by the joint evaluation synthesis - a total of 194 projects related to pastoral development were formulated and US$1.2 billion were allocated. Areas of intervention included animal health and veterinary services, capacity-building, emergency response, rangeland management and policy dialogue. The evaluation finds that FAO's and IFAD's engagement in pastoral development has been significant, but lacks a coherent conceptual framework and systematic direction. There have been positive results in projects seeking to reduce poverty and hunger by introducing innovative solutions in community-based animal health and natural resource management. However, the evaluation finds that there is still considerable confusion between pastoral development and livestock development, and no clear understanding of pastoral systems, including the specificity of pastoral poverty. This has led to a focus on sedentary activities, and a considerable degree of hit-and miss in the results. The recommendations focus on the need for FAO and IFAD to create policies of engagement in pastoral development and build capacity for systemic engagement in pastoral systems through, for example, developing a better understanding of pastoral systems and their relation to dryland economies. Furthermore, risk-management and resilience strategies should be prepared. They need to be highly context-specific and include a distinction between risk management and risk reduction. The evaluation also highlights the need for the two agencies to strengthen advocacy by pastoralists and on behalf of pastoralists.
Community-Based Agricultural and Rural Development Programme
The Community-based Agricultural and Rural Development Programme was one of the biggest and most influential IFAD loan operations in Nigeria. It was based in the North - one of the poorest parts of the country and an area affected by years of conflict, including the recent Boko Haram insurgency. The programme's goal was to improve the livelihoods and living conditions of poor rural communities, with emphasis on women and other vulnerable groups in the seven participating states. The approach to addressing the root causes of poverty in this underserviced and hard-to-reach area was to fill the institutional gap by building a fourth tier of government at village level. Community development associations were formed from elected representatives to take responsibility for planning, implementing and maintaining village-level investments. The programme successfully rolled out the community-driven development approach across many states in the North, and as confirmed by this assessment, created a lasting impact. It has provided poor people with a structure and a space to plan community investments and manage them in a sustainable way. For the Government, this structure filled an institutional vacuum and enabled funding to be channeled into hard-to-reach areas. The model was subsequently adopted by other states and development partners. The community-driven approach was, however, less effective in addressing issues of inequality and transforming power relations within communities. For women, the programme provided a ground-breaking opportunity to participate in the community-level debates, but they were usually passive participants and had a limited role in decision-making.
Bangladesh Country Programme Evaluation
This is the third evaluation of IFAD's country programme in Bangladesh undertaken by the Independent Office of Evaluation. Bangladesh is among the top three recipients of IFAD funding in the Asia and the Pacific region. Since 1979, IFAD has financed 31 projects in the country for a total project cost of US$1.9 billion, of which US$717.2 million are attributed to IFAD. The evaluation found that, overall, the portfolio demonstrated having a positive impact on rural poverty alleviation, in particular with respect to increases in rural household income and assets in project areas, as well as improvements in productivity. This was generated, in particular, from on-farm activities facilitated by the combination of microcredit/seasonal agricultural credit, adoption of improved production and management techniques, marketing support linking with supply/value chain, infrastructure development and in particular to the promotion of gender equality and women's empowerment. Knowledge management is still one weak link of the Bangladesh country programme. Despite recent progress, lessons learned are not systematically captured and shared, resulting in little awareness of IFAD's rich country, regional and global knowledge in Bangladesh. Partnership at the operational level with institutions and key counterpart departments is strong, but did not succeed in harnessing a secured constituency at a strategic policy level.
IFAD's Engagement with Indigenous Peoples: Evaluation Synthesis
IFAD has been financing projects in support of indigenous peoples since 1979, in particular in Latin America and Asia. According to United Nations estimates, there are more than 370 million indigenous people worldwide spread across some 70 countries. They continue to be overrepresented among the poor: while they constitute just 5 per cent of the world's population, they account for 15 per cent of the world’s poor people.
This evaluation synthesis report confirms that IFAD is in a unique position among development agencies to support indigenous peoples' social and economic empowerment. Given its mandate and focus – rural poverty reduction with attention to the vulnerable and marginalized, participatory approaches, community development, empowerment and social inclusion, IFAD has naturally followed a proactive approach to supporting indigenous peoples.
In light of its unique position and comparative advantage, and building on its experiences so far, there is still room to strengthen the consistent implementation of IFAD's policy on engagement with indigenous peoples, in particular at the level of investment projects.
The report highlighted the importance of paying greater attention to key project design elements, such as devising tailored and differentiated approaches to build on the culture, identity and knowledge of the indigenous peoples' communities.
The report also highlighted the importance of enhancing staff understanding on indigenous peoples' issues. IFAD could further strengthen knowledge management in this area, taking advantage of substantial experience, lessons and knowledge of engagement with indigenous peoples.
2015 Annual Report on Results and Impact of IFAD operations
The 2015 Annual Report on Results and Impact of IFAD Operations (ARRI) shows that, overall, IFAD operations are satisfactory and making good contribution to inclusive rural transformation. Projects and programmes had positive results in reducing rural poverty and improving the living conditions of rural poor people, in particular in terms of increased income and assets, better human and social capital and empowerment, and gender equality and women’s empowerment.
Nevertheless, there are areas that can be further improved, such as operational efficiency, monitoring and evaluation and non-lending activities (e.g. policy dialogue, knowledge management and partnership-building).
This year’s learning theme is the sustainability of benefits of IFAD’s operations, which was found to be one of the weakest-performing areas.
Rural Livelihoods Improvement Programme in Attapeu and Sayabouri
The purpose of the Rural Livelihoods Improvement Programme in Attapeu and Sayabouri was to promote economic growth and livelihood improvements among the rural poor, including women and other vulnerable groups in the target area, such as unemployed rural youth and upland ethnic groups. The overall programme approach of supporting the national growth and poverty eradication strategy and working through the line agencies and mass organizations is seen to have been successful, as many outputs seem to have been achieved, especially for infrastructure-building. Considerable investment was put into capacity-building at district and provincial levels to improve participatory and gender-sensitive planning, implementation, assessment and long-term management. On the other hand, with regard to monitoring and evaluation (M&E), IFAD should pay greater attention and provide support at all stages in this area in collaboration with the government including proposing a solid basis for M&E in project design reports, providing support and guidance to project management in strengthening the M&E system and carrying out regular impact surveys.
United Republic of Tanzania Country Programme Evaluation
This country programme evaluation is the second one carried out in the United Republic of Tanzania by the Independent Office of Evaluation of IFAD. It covers the partnership between IFAD and the Government from 2004 to 2014.
The stronger case of effectiveness and impact has been the support to the Agricultural Sector Development Programme (ASDP) in the mainland and, even more, in Zanzibar. In line with national decentralizations policies, local government authorities, particularly districts, were the actual "implementers" of the programme. ASDP introduced a more participatory, "bottom-up" system for preparing local agricultural development plans.
Non-lending activities such as knowledge management, partnership development and policy dialogue have been developed only to a limited extent. On the positive side, partnerships with the Government (mainland and Zanzibar) and the main donors in the agricultural sector have dramatically improved in the past ten years. However, lessons from project experience have not been captured adequately, and the objectives for policy dialogue have been ambitious compared to the limited resources available.
Gente de Valor - Rural Communities Development Project in the Poorest Areas of the State of Bahia
The Gente de Valor - Rural Communities Development Project in the Poorest Areas of the State of Bahia was implemented between 2006 and 2013. The project's development goal was to reduce poverty, especially extreme poverty levels, of semi-arid communities of the State of Bahia. The design of the project was very well adapted to the challenging agro-ecological environment of the area. Among the most important effects of the project were the improved access to water (both for human consumption and for horticulture), access to enhanced production techniques and technology, as well as the significant empowerment of disadvantaged communities, and within communities, of women. Beneficiaries are better aware of social and economic development opportunities in the area and active in trying to pursue them. Smaller-scale and easier-to-manage economic activities promoted by the project, such as the horticultural backyards and the rearing of small ruminants, show good perspectives of economic viability. On the other hand, larger processing plants for agricultural produce require considerable managerial skills, knowledge of markets, as well as working capital. They also require a solid business case analysis and plan. These have not yet been achieved to a sufficient extent.
India: Jharkhand Chhattisgarh Tribal Development Programme
The Jharkhand-Chhattisgarh Tribal Development Programme was implemented between 2001 and 2012 in the two states in India. The programme's objectives were to empower tribal grass-roots associations and enhance their livelihoods through income-generating activities, increasing agricultural productivity and improved land and water use.
Whilst the focus of the evaluation was decisively on assessing impact, it also covered all the evaluation criteria adopted by IOE (e.g. relevance, effectiveness, efficiency, etc.) in order to facilitate a more comprehensive evaluation of the programme.
The evaluation followed quasi-experimental methods, combining quantitative econometric and quantitative techniques. It entailed large amounts of primary data collection from treatment and comparison groups in the two states, which were triangulated with other sources of evidence.
The programme had positive impacts on the target groups. For instance, it reduced the number of people living under the poverty line (US$1.25) within the target group, as compared to those the programme did not support. In addition, it increased incomes and paddy productivity for members of the target group. Notwithstanding the above, the impact evaluation finds that the programme's design was too complex, encompassing many activities covering two states under a single loan. In general, more attention to diversification of the productive base of the rural poor and a sharper targeting would have contributed to enhanced results.
Livelihoods Improvement Project in the Himalayas
The Livelihoods Improvement Project in the Himalayas was implemented between 2006 and 2013, aimed to improve the livelihoods of vulnerable groups in a sustainable way by promoting greater livelihood opportunities and strengthening the local institutions concerned with livelihood development. Implemented in both Meghalaya and Uttarakhand, the project was successful in its sequencing of activities for the engagement of women, firstly, reducing drudgery, then providing empowerment activities through group formation (social and financial), and then building their social capital to engage in livelihood activities. However, a number of challenges affected final results, such as the fact that the project was implemented in two non-contiguous states; there was no cross-learning between the states; and a single budget was allocated for supervision and implementation support, which had to be split between the two states.
Sivas-Erzincan Development Project
The Sivas-Erzincan Development Project in Turkey, implemented between 2003 and 2011, aimed to increase agricultural productivity and income levels of the rural poor, expand employment opportunities and encourage smallholder initiatives. The project made an important contribution in terms of supporting the communal infrastructure including new irrigation schemes and improvements to dairy-related activities led to a significant increase in agricultural productivity and income levels for the rural poor in the less developed parts of the project area. The project also financed non-repayable grants which created a secure and steady income from dairy production. Despite important steps taken and contributions made, the overall achievements fell short of the set objectives. The late and incomplete establishment of properly functioning monitoring and evaluation systems led to severely hindering the analysis of project results. Training of project support staff also needs attention.
Community Development Programme
The programme covered Azad Jammu and Kashmir in Pakistan between 2004 and 2012. Its achievements are most visible in terms of improving access of rural communities to social and economic infrastructure, and strengthening the overall capacity of community organizations and the skills of their individual members. The programme impact is most pronounced in the area of human and social capital and empowerment. Despite important steps taken and contributions made by the Government of Azad Jammu and Kashmir to sustain the benefits of the programme, one of the main issues concerning sustainability is that the participatory approach has been "project-centred" and has not been mainstreamed into government's regular development planning, budgeting and service delivery. Programme implementation progress in the initial years was hindered not only by the 2005 earthquake but also by various implementation and management issues, in particular, a continual turn-over in the programme director's position. However, after the mid-term review, the programme made reasonable progress due to a number of factors, including stability of leadership positions in the Programme Management Unit and direct supervision and closer follow-up by IFAD.
Programme for Sustainable Development in Rural Mountain Areas
The goal of the Programme for Sustainable Development in Rural Mountain Areas of Albania was to increase household incomes through three objectives: additional resource mobilization in and for the mountain areas; accelerated economic growth and poverty reduction; and strengthened abilities of local institutions to support private- and public-sector investment. The project was able to identify relevant value chains and key constraints to enhancing incomes. Experiences in community empowerment and institutional development were also promising at local level. Some farmers and entrepreneurs successfully used project grants or loans to expand their operations, resulting in higher incomes and the creation of jobs. However, the project did not meet expectations at the national level. In particular, an effective mountain area development agency has not emerged, nor has a sustainable rural lending institution serving small rural farmers or entrepreneurs. The project design did not adequately take into account critical issues and recommendations raised during a former evaluation by IOE and IFAD design reviews. A lack of an appropriate monitoring and evaluation system and the absence of a midterm review limited corrective measures that could have been introduced.
IFAD’s Engagement in Fragile and Conflict-affected States and Situations
This is the first corporate-level evaluation on IFAD's engagement in fragile and conflict-affected states and situations (FCS) undertaken by the Independent Office of Evaluation (IOE). It reflects IFAD's growing involvement in such contexts, and growing global interest in FCS. The evaluation focuses on IFAD's work with FCS over a 10 year period from 2004 to 2013.
The evaluation found that IFAD has a critical and distinct role to play in addressing the problems of fragile states which, in turn, are key to achieving a range of United Nations Sustainable Development Goals including the elimination of poverty, the promotion of sustainable agriculture and productive employment and the building of peaceful and inclusive societies.
The evaluation shows that there have been significant improvements in overall project achievement, project effectiveness, IFADÃ's performance as a partner and rural poverty impact in fragile countries. At the same time, the evaluation underlined the importance for IFAD to develop a new policy/strategy for its work in fragile situations, which would ensure even more customized approaches for better development effectiveness