American Evaluation Association (AEA) Conference: Evaluation 2018
The conference took place from 31 October to 3 November in Cleveland, Ohio, United States of America. It included more than 40 professional development workshops, featured 700 evaluation presenters, and served to connect a community of more than 3,000 evaluators.
The Deputy Director of the Independent Office of Evaluation of IFAD (IOE), Fabrizio Felloni, participated in the Panel Session "Experiences in evaluating international development support to agricultural value chains for poverty reduction".
The session was dedicated to two corporate-level evaluations on pro-poor agricultural value chain development recently conducted by two major international development organizations, the African Development Bank and the International Fund for Agricultural Development. The presentations outlined the definition of agricultural value chains and the methodological challenges in: (i) assessing the performance and the emerging results of development interventions that are at their initial stages; and (ii) in aggregating findings and explanatory factors. The presentations highlighted key factors for success of failure and to what extent and under what conditions, supporting value chain linkages can effectively benefit smallholder and poor farmers and rural producers.
Visit the AEA website for more information.
Global Impact Investing Network (GIIN) Investor Forum 2018
The impact investing movement is reshaping the future of the financial markets as people are increasingly considering the environmental impact of their investments in addition to the financial returns. The GIIN Investor Forum brought together over 1,000 impact investors to help shape the future of impact investing.
The director of the Independent Office of Evaluation of IFAD (IOE), Mr Oscar A. Garcia, participated in the event, which took place on 29-31 October 2018 in Paris.
Please visit the event website for more information.
Burkina Faso Country Strategy and Programme Evaluation - National Workshop
The Government of Burkina Faso and the Independent Office of Evaluation of IFAD have co-organized a national workshop - in collaboration with IFAD’s West and Central Africa Division - to discuss the main issues emerging from the evaluation of IFAD-funded country strategy and programme in Burkina Faso, as a final step in the evaluation process.
The event took place on 23 November 2018 in Ouagadougou. Around 100 participants attended the workshop, including senior representatives from the Government of Burkina Faso, IFAD Management and staff, multilateral and bilateral development organizations, project management staff of IFAD-funded projects and programmes, academics and research institutions, private sector and foundations, non-governmental organizations, and selected resource persons.
The objectives of the workshop were to:
- discuss the main issues emerging from the Burkina Faso Country Strategy and Programme Evaluation;
- provide inputs for the preparation of the evaluation’s Agreement at Completion Point (ACP); and
- discuss the opportunities and challenges of the IFAD-Government of Burkina Faso partnership in the future, while considering the evolving development needs and priorities of the country.
Advancing towards the SDGs based on evidence – evaluation perspectives. Reflections from Latin America and the Caribbean on Voluntary National Reviews (VNRs) and legislative work
The webinar, entitled Advancing towards the SDGs based on evidence – evaluation perspectives. Reflections from Latin America and the Caribbean on Voluntary National Reviews (VNRs) and legislative work was organized by the Latin American and Caribbean Network of Evaluation (ReLAC) as part of the Week of Evidence for Latin America and the Caribbean from 22 to 26 October 2018.
The objectives were to generate a dialogue on the importance of the responsible use of evidence derived from evaluations in political decisions. It focused on the National Voluntary Reports (2016-2017), analysing the inclusion of evidence from evaluations and the extent to which the information contained in these reports is sufficient to inform progress towards the SDGs. The webinar also emphasized how the use of evidence in parliamentary work is key to parliamentarians' decision-making process.
Panelists of the event were: Oscar A. García, IOE Director, and Ivonne Acuña, Deputy of the Legislative Assembly of the Republic of Costa Rica. The event was moderated by Silvia Salinas, Coordinator, ReLAC. Over 55 people were connected through webEx.
The event was in Spanish.
2018 ARRI learning event
On 26 October, the Independent Office of Evaluation (IOE) held a learning event on the 2018 Annual Report on Results and Impact of IFAD Operations (ARRI) including its theme on targeting the rural poor.
The objectives of the learning event were to share and further discuss the findings presented in the ARRI 2018, with a particular emphasis on improving IFAD's performance and reaching the rural poor.
The 2018 ARRI presents a synthesis of IFAD’s performance and highlights results and systemic issues from independent evaluations conducted in 2017. The quantitative analysis draws on ratings from 320 evaluations conducted since 2002.
The event was well attended by close to 60 participants, including IFAD staff and representatives from FAO, WFP and IFAD Executive Board. The keynote speaker was Kevin Watkins, Chief Executive, Save the Children UK. Other guest speakers were Margarita Astralaga, Director, Environment, Climate, Gender and Social Inclusion Division; Khalida Bouzar, Director, Near East, North Africa and Europe Division; Lisandro Martin, Director, West and Central Africa Division; Fabrizio Felloni, Deputy Director, IOE and Guoqi Wu, Associate Vice-President, Corporate Services Department.
Tunisia Country Strategy and Programme Evaluation – National Workshop
The Ministry of Development, Investment and International Cooperation, the Ministry of Agriculture, Hydraulic Resources and Fisheries, and the Independent Office of Evaluation of IFAD (IOE) co-organized a national workshop for the Country Strategy and Programme Evaluation (CSPE) conducted by IOE in the course of 2018, as a final step in the evaluation process.
The national workshop took place on 23 October 2018 in Tunis. Around 50 participants attended the workshop including the honourable Minister of Development, Investment and International Cooperation, representatives of the Government of Tunisia, representatives of multilateral and bilateral development organizations (including the European Union and the Argentinian cooperation), academics and research institutions, private sector (including microfinance institutions), the Farmers Union and Farmers Syndicate, and selected resource persons.
The objectives of the national workshop were to:
- Share and discuss the key findings and recommendations of the Country Strategy and Programme Evaluations; and
- Provide an opportunity for reflecting on the main aspects and priority areas for the forthcoming Tunisia Country strategic opportunities programme (COSOP).
IFAD's operations support better living conditions for 50,000 rural Tunisian households
You can read the press release in Arabic here.
Press release No.: IFAD/71/2018
IFAD has invested in rural people for 40 years, empowering them to reduce poverty, increase food security, improve nutrition and strengthen resilience. Since 1978, we have provided US$20.4 billion in grants and low-interest loans to projects that have reached about 480 million people. IFAD is an international financial institution and a specialized United Nations agency based in Rome – the UN’s food and agriculture hub.
The Independent Office of Evaluation (IOE) conducts evaluations of IFAD-financed policies, strategies and operations to promote accountability and learning. The main purpose is to contribute to improving IFAD's and its partners' performance in reducing rural poverty in recipient countries. IOE's independent evaluations assess the impact of IFAD-funded activities and give an analysis of successes and shortcomings – to tell it the way it is – as well as identify factors affecting performance. Based on the key insights and recommendations drawn from evaluation findings, IOE also shares IFAD’s knowledge and experience in agriculture and rural development with a wider audience.
World Bank event - Book Launch: Evaluation for Inclusive and Sustainable Rural Transformation
On Thursday 23 October, the World Bank hosted the launch of the book Evaluation for Inclusive and Sustainable Rural Transformation: World Bank Series on Evaluation and Development, Volume 9. The book was co-authored by Oscar A. Garcia, Director of the Independent Office of Evaluation of IFAD, and Osvaldo N. Feinstein, Professor in the Evaluation Master’s Programme of the Complutense University of Madrid.
The event was opened by Sophie Sirtaine, Director, Strategy and Operations at the Independent Evaluation Group (IEG), World Bank. During the event, Oscar A. Garcia, IOE Director, and Osvaldo N. Feinstein delivered a presentation of the book. A review was provided by Martha Elena Federica Bárcena Coqui, Ambassador Permanent Representative of the United Mexican States to IFAD.
The event took place on 23 October in Washington D. C., United States, at the Library of the World Bank.
How can IFAD reach the rural poor and leave no one behind?
Findings from the 2018 Annual Report on Results and Impact of IFAD Operations (ARRI)
By Chitra Deshpande, Senior Evaluation Officer, Independent Office of Evaluation of IFAD
In the context of the 2030 Agenda for Sustainable Development, targeting is central to IFAD's mandate of rural poverty reduction. It is also a key principle of engagement for the current strategic framework. From 2007, IFAD's Targeting Policy has made poverty targeting a requirement in all IFAD investments. Given this importance, the Independent Office of Evaluation of IFAD conducted a review and identified five findings on IFAD's poverty targeting approaches which are presented in the 2018 ARRI's learning theme chapter and issues paper "Targeting the rural poor".
Our first finding is that although IFAD is known for reaching the "poorest of the poor", there is a lack of agreement within IFAD on our target group and the strategies needed. The Targeting Policy states that IFAD's target group is "rural people living in poverty and experiencing food insecurity in developing countries." It goes on by adding that IFAD "proactively strives to reach extremely poor people (as defined by Millennium Development Goal 1) who have the potential to take advantage of improved access to assets and opportunities for agricultural production and rural income-generating activities." Since 2008, IFAD has also increasingly engaged in market-oriented agriculture or value chain approaches directed towards less poor groups with greater commercial potential. Therefore, the ARRI highlights the importance of finding a balance between market-oriented and poverty-focused projects and the need to tackle the targeting challenges that subsequently arise.
Secondly, we found effective targeting strategies are based on differentiated poverty analysis of excluded groups such as women, youth, indigenous peoples or pastoralists. They are based on good contextual analysis so that targeting strategies are realistic, clear and practical. This is especially important in fragile contexts. With regards to gender equality and women's empowerment, gender strategies with specific targets are required. For youth, community-driven development with rural enterprise activities have proven effective. And for indigenous peoples, strategies should be culturally-sensitive and recognize and appreciate their different knowledge systems.
Finding 3 highlights the importance of credible poverty data, supported by monitoring through supervision and implementation support. Systematic monitoring allows targeting strategies to be flexible and responsive to a rapidly changing world. For example, in Mauritius a project design failed to recognize the rapid economic transition and many targeted households opted for employment in manufacturing and service sectors rather than stay in agriculture. In Cambodia, a project design initially favoured poor landowners, this was adjusted at mid-term when the project introduced approaches to identify poorer households. This resulted in identifying and issuing ID cards to the most vulnerable families through which they gained free access to government services.
Finding 4 confirms that reaching the poorest people and the "last mile" is costly but essential to achieve IFAD's mandate and contribute to the Sustainable Development Goals. More time and resources may be needed to design and implement projects that target marginalized groups living in remote areas as was the case in a project in Brazil which used demand-driven participatory approaches. Pursuing efficiency can also push targeting away from the poorest or those in remote areas.
Finally, Finding 5 highlights the importance of government commitment and partnership to reach extremely poor rural people. Government commitment to prioritizing rural poverty results in systematizing poverty targeting data and engaging in policy dialogue. For example, in Nepal the forest leaseholder approach was eventually integrated into the national policy. Partnering with international, regional and local agencies is important for policy dialogue. It is also a means of providing basic needs to extremely poor people, as was evident in the IFAD partnership with the Belgian Survival Fund in Sub-Saharan Africa.
IFAD's targeting policy acknowledges that poverty is context-specific and multi-dimensional. Therefore, its implementation requires greater investment in capacity for the required poverty data, differentiated analysis and responsive monitoring and supervision to develop poverty targeting strategies that are realistic, clear and flexible for different target groups and changing country contexts.
Based on these findings, the ARRI recommends revising the 2007 Targeting Policy and guidelines in order to establish greater clarity on who IFAD interventions should target. This is especially important given the emphasis on reaching the "poorest and most vulnerable people in each country" in the IFAD11 Consultation report and the 2030 Agenda commitment of "leaving no one behind." If IFAD is serious about targeting extremely poor people living in remote rural areas, then the Targeting Policy needs to clearly establish them as IFAD's primary target group. This will strengthen country programme managers' position in negotiations with governments and other partners and allow IFAD to more consistently "walk the talk" and truly realize the IFAD11 commitments.
Evaluation criteria: who wins and who loses from development interventions?
By Oscar Garcia, Director, Independent Office of Evaluation of IFAD
The evaluation criteria developed by the Organisation for Economic Co-operation and Development - Development Assistance Committee (OECD-DAC) in 1991 influenced the practice of evaluation in a significant way. Relevance, effectiveness, efficiency, sustainability and impact are amply used, validated and recognized internationally. They are a cornerstone of the global architecture to evaluate development assistance and have been instrumental to improve accountability and learning. The standardized approach allows for aggregation and meta-analysis. What is interesting about the evaluation criteria is their broad applicability. They are useful to assess development interventions in any sector, in health, education, industry, trade, social protection, energy or agriculture.
After many decades of use, the evaluation criteria need some updates. There are three main sources of criticism. The first one comes from the limited scope, arguing that the criteria were developed with projects in mind. Currently, more complex development interventions are needed in policies, programmes or strategies to achieve the expected development outcomes such as eradicating extreme poverty, adapting to climate change or ending hunger. In other words, how the evaluation criteria can be useful in more complex development contexts, adopting a systemic approach. The second one refers to the definitions and their need to be updated, for instance, on the different dimensions of relevance. The third one comes from the rigid application of criteria that may highjack their potential to be used in a variety of contexts. The use of criteria without sufficient consideration of the context in which the evaluation takes place, has been identified as a constraint.
What needs to be done? The OECD DAC evaluation criteria can (a) be kept as they are; (b) be transformed, including by updating their definitions; or (c) be expanded, by adding new criteria.
I am in favour of a combination of transforming them and adding new criteria. The OECD DAC evaluation criteria should continue orienting the evaluation practice and I would not question the importance of keeping them. I would simply adjust the definition of relevance to include the dimension of the appropriateness of design and would be more explicit on the indicators to measure efficiency. The main proposal I have, however, is to add a new criterion.
I propose to add coverage as an evaluation criterion to assess development assistance. Coverage was previously developed by Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) as part of the evaluation criteria for humanitarian action. It was understood as the extent to which major population groups were reached by humanitarian action.
In the context of the 2030 Agenda for Sustainable Development, where the ambitious goals demand to reach out to marginalized groups of population in order to eradicate poverty, end hunger and spread prosperity, the disaggregated data on every development initiative needs to come clearly to the fore. Based on IFAD's experience, identifying more clearly the target population and their differentiated needs, which may include indigenous peoples, pastoralists, people with disabilities, women or youth, improves the soundness of the interventions. Adding coverage to the set of evaluation criteria would allow to respond to the political economy question of who benefits and who loses from the development interventions. It has the advantage of universal application. Who benefited from the initiative can be asked in every sector and would be valid according to the initiatives' objectives. Which population groups were reached out and which were left out will answer, in a standardized and systematic way, one of the main concerns about the Agenda 2030, namely to not leave anyone behind.