Kenya: Smallholder Horticulture Marketing Programme
The Smallholder Horticulture Marketing Programme mainly targeted smallholder horticultural farmers that produce for the domestic market, input suppliers (stockists), produce traders, transporters, and processors. It reached about 21,311 direct households in 14 districts, and 36 per cent of beneficiaries were women.
The evaluation was conducted using a quasi-experimental approach, which combines econometric and qualitative techniques to estimate the impact of the project. A total of 1500 households of beneficiary and comparison groups were interviewed for the evaluation. Two measures of food security were employed to assess the impact on food security of beneficiaries – the Household Food Insecurity Assessment Score and the Household Dietary Diversity Score.
The project showed positive results on horticultural producers’ incomes and their food security. This was primarily realized through the production node of value chains, which employed a district-based approach. Training was provided to commercial village groups, however there was a greater impact on agronomic practices than on marketing knowledge.
The report found that the lack of trust among group members was the most common denominator in explaining the less-than-desired outcomes in commercial villages. Issues of lack of accountability and poor governance also acted as barriers to successful collaboration in groups dedicated to commercialization. Additionally, the effects of the devolution from central to county governments were most visible in relation to the market infrastructure aspect. Several market structures, built using the programme funds, in fact, were not functioning.
The evaluation recommends adopting an integrated approach and a proper sequencing of activities in value chain-related interventions.
Georgia: Agricultural Support Project
The overall objective of the Agricultural Support Project was to increase the incomes of the rural poor through commercially viable agricultural and rural enterprises. The project reached about 6,000 households in regions with high incidences of rural poverty but with high productive potential in agriculture.
The evaluation followed an ex post quasi-experimental approach and combined econometric and qualitative techniques to determine the impact of the project on its beneficiaries. It deployed two innovative approaches: (1) genetic matching method for matching the treatment group with the comparison group (a total sample size of 3,100 respondents was selected, which gave superior results as compared to the propensity score matching method); and, (2) satellite imagery to compute the normalized difference vegetation index for comparing the change in vegetation cover between treatment group farm plots and comparison group farm plots before and after the rehabilitation of irrigation schemes.
The project's activity of rehabilitating irrigation canals was successful in bringing irrigation water to the beneficiaries, although the regularity of the water supply remained an issue. The project has triggered revitalized interest in agriculture, encouraging other agencies such as the World Bank to scale up the neglected irrigation schemes. However, the impact evaluation demonstrated that interventions under the major project component, small-scale infrastructure rehabilitation, did not lead to statistically significant changes in the incomes, assets or food security of beneficiaries as compared to non-beneficiaries. The reasons range from the delayed start of the irrigation scheme rehabilitation to the lack of rehabilitation of all tertiary canals.
Republic of Mozambique: Sofala Bank Artisanal Fisheries Project
The Sofala Bank Artisanal Fisheries Project focused on community development, fisheries development, markets, financial services and policy support. It targeted about 500,000 people, including 26,000 direct beneficiaries in 290 fishing communities along the coast.
Whilst the focus of the evaluation was decisively on assessing impact, it also covered all the evaluation criteria adopted by IOE (e.g. relevance, effectiveness, efficiency, etc.) in order to facilitate a more comprehensive evaluation of the programme. The evaluation followed quasi-experimental methods, combining quantitative econometric and quantitative techniques with the aim of attributing impact to the project. The evaluation designed an impact survey to collect primary quantitative data, which was administered to a sample of 1,028 households including beneficiary and non-beneficiary households. The quantitative part of the evaluation was complemented by a rich qualitative analysis to allow for a more comprehensive understanding of the processes of change induced by the intervention.
SBAFP was an important milestone in the development of the artisanal fishery sector. This is attributed to its integrated livelihood approach, which delivered tangible results beyond fishery development in remote and complex fishing areas. The project had remarkable impacts at the household, institutional and policy levels. This includes better incomes and assets among beneficiaries, enhanced human and social capital, improved access to social and market infrastructures, as well as better participation in grass-roots institutions and in particular in savings and credit groups. SBAFP made an important contribution to policy formulation and legislation favouring the artisanal fishery subsector and helped strengthen institutions in the subsector. The development of the Plano Estratégico para o Sector da Pesca Artesanal (PESPA 2006-2016), which is planned to be renewed in collaboration with the World Bank, stands out as one of the project’s highest achievements.
However, more could have been done to achieve greater results and impacts. Despite the key role that women play in the fishery value chain, the project design did not include a strategy for gender mainstreaming, thus constraining results on gender equality and women’s empowerment. The support to market access remained mainly confined to developing infrastructure and improving post-harvesting practices. In particular, the inadequate attention to private sector linkages and engagement, business counselling services, and marketing of fishing products limited wider impact and transformation of the artisanal communities. Finally, the weaknesses in the monitoring and evaluation system and availability and quality of data impinged on the ability to assess and attribute impact to the project.
India: Jharkhand Chhattisgarh Tribal Development Programme
The Jharkhand-Chhattisgarh Tribal Development Programme was implemented between 2001 and 2012 in the two states in India. The programme's objectives were to empower tribal grass-roots associations and enhance their livelihoods through income-generating activities, increasing agricultural productivity and improved land and water use.
Whilst the focus of the evaluation was decisively on assessing impact, it also covered all the evaluation criteria adopted by IOE (e.g. relevance, effectiveness, efficiency, etc.) in order to facilitate a more comprehensive evaluation of the programme.
The evaluation followed quasi-experimental methods, combining quantitative econometric and quantitative techniques. It entailed large amounts of primary data collection from treatment and comparison groups in the two states, which were triangulated with other sources of evidence.
The programme had positive impacts on the target groups. For instance, it reduced the number of people living under the poverty line (US$1.25) within the target group, as compared to those the programme did not support. In addition, it increased incomes and paddy productivity for members of the target group. Notwithstanding the above, the impact evaluation finds that the programme's design was too complex, encompassing many activities covering two states under a single loan. In general, more attention to diversification of the productive base of the rural poor and a sharper targeting would have contributed to enhanced results.
Democratic Socialist Republic of Sri Lanka: Dry Zone Livelihood Support and Partnership Programme
The Independent Office of Evaluation conducted its first Impact Evaluation in 2013 of an IFAD-funded project: the Sri Lanka Dry Zone Livelihood Support and Partnership Programme. The Impact Evaluation entailed, inter-alia, extensive primary data collection and analysis, including a quantitative survey of over 2,500 households. The survey covered households that benefitted from project support and those that did not (the latter being the control group). The evaluation found that the project played an important role in exposing smallholder farmers to new crops and improved agricultural techniques and can be considered as a pioneering intervention that built momentum, human capital and experience. However the effects on household assets and expenditures were mixed compared to the project’s monitoring and evaluation data and much remains to be done to consolidate the emerging dynamics.