Qinling Mountain area Poverty-alleviation Project (2010)

China  
December 2010

Interim Evaluation

Introduction

Evaluation objectives. An interim evaluation of the Qinling Mountain Area Poverty-Alleviation Project (QMAPAP) in China was conducted in 2008. The main objectives were to: (i) assess the performance and impact of the project; and (ii) generate a series of findings and recommendations to guide the Government of China (GOC) and the International Fund for Agricultural Development (IFAD) in financing a follow-on project. The evaluation covered all components of the project and not only those items financed by IFAD.

Poverty situation. Poverty in China fell substantially from 360.6 million people in 1990 to 101.2 million in 20061.  Yet, in that same year, 128 million Chinese were still estimated to live on below a dollar per day. The poorest communities in China are mainly found in remote and resource-poor areas in the western and interior regions, often without sufficient access to clean water, arable land or adequate health and education services. The QMAPAP focussed in one of these remote areas: the Qinling Mountain range which covers areas in the Shaanxi and Hubei provinces. The QMAPAP aimed to target 310,000 households in the nine poorest counties of the two provinces and reach about 90 per cent of the village population in the 128 targeted towns.

Project description. The project aimed to achieve a sustainable increase in productive capacity, and offer increased access to economic and social resources for the targeted towns. The project design included five components: agriculture development, rural infrastructure, rural financial services, social development, and project management. The management structure for the project built on existing local Government structures at Provincial, Prefecture, County and Town level. Project Leading Groups (PLGs) were established at each level to facilitate coordination between project partners.

The total cost of the project was estimated to be US$106.3 million2 , of which IFAD's loan contribution was US$29 million3. Other contributors were: US$10.4 million in grant co-financing from the World Food Programme (WFP), US$4.1 million from project participants and US$62.6 million from the Chinese Government. The IFAD loan largely supported a credit fund to stimulate rural financial services plus a contribution towards agriculture development and project management.

Project design and timeline. The project was formulated during April 1999 and approved by the IFAD Executive Board in December 1999. The Loan Agreement was signed in February 2000, but loan effectiveness was not declared until August 2001. The delay was largely due to difficulties in the establishment of partnership arrangements with WFP and finalising the sub-loan agreement for the rural finance component which were conditions for loan effectiveness. The loan closed as scheduled in March 2008.

Summary of implementation results

Overall, the project has performed well. Of the total financial allocation, 93 per cent has been expended, with expenditure per component being close to the targeted amounts. The implementation results of the project have been in accordance with targets, with most key objectives being met or exceeded. The initial delays in project effectiveness created implementation difficulties which reduced the potential synergy of rural credit with other components early in the project. However, there were substantial improvements in overall performance after the Mid-Term Review (MTR) in 2005.

The changing context of China had a major influence on the performance. The effects of policy and market changes during the project period positively influenced farming and labour practices. The project aligned well with the policy changes overall and contributed towards increased access to markets and credit funds. The project investments combined with Government investments to build local capacity and on-ground improvements in infrastructure, agricultural practices and community capacity. The project inputs not only created development opportunities within the towns and villages but also increased access by villagers in remote area to economic opportunities in other areas.

Agriculture development. Terracing, land levelling and land improvement have been carried out in the majority of project villages. Irrigation facilities were improved with a total investment of US$19.8 million. Productivity of food crops increased by an average of approximately 20-50 per cent per household, both through introduction of higher yielding varieties, diversification and improved soil and farm management. Forest coverage rate has greatly increased with both commercial tree crops and environmental trees to protect watershed areas.

Infrastructure development. Construction of infrastructure has been successful, with most types of infrastructure attaining the targets set. The new and improved roads and bridges reduced transport costs and increased access to markets.  Irrigation and flood protection resulted in increased crop yields. Potable drinking water systems have improved quality of life through substantially reduced time in carrying water, as well as associated health benefits. The expenditure for rural infrastructure was slightly higher than estimated, partly due to high demand from the communities and rising prices per unit of infrastructure during the project period. The biogas units built by the project served as an effective pilot for government projects and the number of biogas units in the project area has substantially increased through other government and private investment.

Rural financial services. The project credit fund was channelled through the existing network of Rural Credit Cooperatives (RCCs) which were the best option for rural lending at the time of design. Despite initial delays, around 85 per cent of the loan funds were disbursed. Local RCCs operate independently so the performance of RCCs varied considerably. An estimated 459,300 loans were disbursed by the time of the evaluation compared to the target of 572,000 (80 per cent accomplishment). This result was partly due to late commencement of credit activities so that the expected number of loans was not attained. Insufficient training of RCC and project staff regarding credit promotion and management was also a contributory factor. Furthermore, credit demand changed during the project. For some villagers the project loan amounts were insufficient for their investment needs. Others were able to borrow from friends and relatives so had less demand than expected for formal credit. The project did respond to the changing demand after MTR by increasing maximum loan amounts from CNY 5,000 to 7,000, but demand in the villages by the end of the project was still for higher loan amounts.

An overall loan repayment rate of 92 per cent has been estimated but again, different RCCs had varying definitions of what constituted delinquency. There was also merging of banking and local customary practices which led to different loan conditions and lending practices being applied in each area. Nevertheless, the level of credit availability has increased in all areas and clients who joined RCCs during the project have largely remained as active clients. The project-supported loans did target the poor and have contributed to stimulation of the village economy.

Social development. Participation in the project was achieved through the Village Implementation Groups (VIGs). The VIGs comprised Village Committee members plus women's and farmer's representatives. The aim of establishing the VIGs was to increase the participation of village members in local governance in general and in identifying priority activities for investment in particular. The VIGs were effective during the project period and were instrumental in coordinating installation of infrastructure, training programmes and other activities. Few remain active beyond the project period in the same format as during the project but the existing Village Committees have generally retained the remaining roles of the VIGs. The social development activities included training for women which was highly appreciated, particularly the technical training.

The participation of women in project activities grew as the proportion of migration labour for men increased. Women's status in the household increased as they were more frequently left to be the key workers and decision makers in the village in the absence of the men. 

Project management. Project Leading Groups and Project Management Offices (PMOs) were established and operated effectively. The organizational structure had two provincial PMOs, one for each province.  The structure operated reasonably well and was appropriate to maximise gains through capacity-building of local government. In the early years of the project, the IFAD processes were cumbersome and not clearly understood by the PMOs. After MTR these issues were addressed, formats were streamlined and processing time reduced. There has been a persistent issue with the design and operation of the monitoring and evaluation system which has created difficulties in data analysis and been a barrier to effective project management.

Project performance

Relevance. The QMAPAP design had high relevance to the project area and target groups. The project objectives were consistent with IFAD and national agriculture and rural development strategies and policies.  The project inputs were relevant and based on local needs, and the balance of cost allocation for each component generally matched the highest priority demands for project support in the project area. The project operated with a demand driven targeting approach which was managed at the local level. As a result, the evaluation found that results and approaches varied across the project area.

The amount of investment per village varied to respond to the highest needs identified by the villages. Nevertheless, some villages were less able to access enough resources and thus key priorities in their isolated sub-village groups were not fully addressed.

Effectiveness. The project was timely given the wider economic opportunities in the country, enabling poor families to take advantage of wider contextual opportunities. The project has effectively reached its objectives largely due to the combination of positive policy advances in the rural areas and project investments. The villages appeared to have benefited substantially from project inputs. There were elements of project operation that did not reach their full potential due to natural calamities, staffing issues and insufficient reach to the most remote households. Nevertheless, overall the quality of life in the villages has substantially improved in all key indicators. The credit component processes could have been implemented in a timelier manner so that greater synergy could have been achieved with other project initiatives.

Efficiency. The project exceeded the Appraisal Economic Internal Rate of Return (EIRR) estimates of 17.8 per cent; with estimated EIRRs of 27 per cent for Hubei and 24.5 per cent for Shaanxi.  QMAPAP also compares favourably to other similar World Bank (China) and other IFAD projects, having achieved relatively low costs per beneficiary and operating costs. The project has not exclusively achieved these benefits but has been an important contributor to improving livelihood security as well as contributing to an improvement in overall prosperity in the project areas. In general the project was assessed as efficient.

Rural poverty impact

Household income and assets have shown substantial increase. Migration work has brought the biggest increase in income for many households. While not directly attributable to the project, QMAPAP initiatives facilitated access to employment. In terms of human and social capital and empowerment, participation by both men and women in skills training allowed access to new technology and broadened knowledge of changing agricultural techniques. Involvement of women in village decision-making increased markedly. Some broader self-mobilisation and self-organization of people occurred in some villages but in general villagers' organizations are still underdeveloped. 

As agricultural productivity increased, surplus food production has increased and food security is no longer a pressing issue in the project area. Agricultural production has been greatly commercialized with large areas of cash crops planted. The rural finance component played an important part in financing new or expanded activities.

Training and livestock services had positive impact on livestock productivity, particularly for lower income households. There are some concerns regarding adoption of cash crops which are price sensitive by lower income households who are less resilient to market fluctuations. Natural resources have been more effectively utilized for food, cash and tree crops. Forest coverage has been greatly increased.

There are some negative impacts on the environment caused by intensive farming, especially for cash crops and fruit tree production because chemical fertilizers and pesticides are increasingly used.

The most significant improvements in institutions and policies came from the good cooperation between different partners. The capabilities of local public institutions in servicing the poor improved.  Yet, the project had little impact on the RCCs. The project approach conflicted with their strategy to avoid risks and to reduce operational costs. There was little policy dialogue on issues related to the project either in design or implementation. Overall, the project demonstrated strong achievement in terms of rural poverty impact.

Sustainability and innovation

Project initiatives have been a strong vehicle for sustainable change in all levels of the project. The benefits generated through infrastructure and agricultural development are likely to continue, and the credit will remain available to clients linked to the RCCs during project implementation. However, there was no formal exit strategy, and roles and responsibilities of the local government in follow on activities are not clear in all areas, which would be the key factor for sustainability as most project operations have been absorbed in existing Government processes. There is concern regarding the sustainability of some infrastructure facilities due to lack of maintenance mechanisms. The rural financial services initiated through the project are liable to continue at current levels, however there are concerns regarding the continuation of the RCCs reaching out to new poor clients once the IFAD funds are no longer directly supervised. There were few major innovations in the project, but some minor innovations, such as linkage of skills training and extension services with RCC loans, piloting of biogas units, VIGs supporting RCCs in identifying potential borrowers and registration of irrigated facilities as common ownership helped to improve project performance.

Performance of partners

The performance of IFAD was considered as low in the first half of the project. Delay in implementation of the IFAD-supported activities caused implementation issues. The Government pre-financed some activities to be financed by IFAD to avert delays in key implementation activities. The rural finance activities commenced late affecting the potential synergy of the project. The delays were compounded by processes which were considered by the project management to be unclear and overly time-consuming. At MTR, IFAD took action on the concerns, particularly reducing unnecessary documentary requirements and increasing loan ceilings. As a result there were clear improvements in the latter half of the project. The opening of the IFAD Coordination Office in Beijing was considered as very valuable in improving coordination between IFAD and GOC. As cooperating institution for project supervision, United Nations Office for Project Services (UNOPS) performed moderately satisfactorily, conducting regular supervision missions but which were considered to be overly focussed on credit and gender rather than the overall context of the project. The GOC provided the required financial and human resources for the project in an effective manner. The various ministries and line agencies (at national, provincial and local levels) involved in project implementation performed well with strong coordination and efficient use of resources. The role of the Women's Federations was well-integrated in project operation and their capacities of training and women supporting were enhanced.

However, the performance of the RCCs was rated as moderately unsatisfactory due to the delays and difficulties in delivering credit services to target groups.

Conclusions

In line with the assessment, the project's overall achievement has been rated as satisfactory. The ratings in relation to each criterion are presented in Table 1, and the ratings for partners' performance are presented below

Summary of project performance and impact

Evaluation criteria
Project evaluation ratings

A. Core performance criteria

Relevance
5
Effectiveness
5
Efficiency
5
Project Performance
5
B. Rural poverty impact
5
Household income and assets
5
Human and social capital and empowerment
5
Food security and agricultural productivity
5
Natural resources and the environment
5
Institutions and policies
4
C. Other performance criteria
 
Sustainability
4
Innovation, replication and scaling up
4
D. Overall project achievement
5
 

Ratings of partners performance

Partners Project evaluation ratings
IFAD
4
Government
5
Cooperating institution (UNOPS)
4
WFP
5
RCCs
3
WF
5
 

Changing context and a shifting poverty definition.  Much policy change occurred in the project areas during the project period.

Policy changes had substantial influence on the pattern of farming, stimulating on-farm investment and a turn towards more commercial farming.  The project was only a partial contributor to the advances in the villages but it was a timely intervention that accelerated development and provided an opportunity for very poor households to benefit from the economic opportunities available. The poverty targeting in the project was initially appropriate because it assisted villages to classify households into the better-off, poor, poorer and very poor categories, with project clients being selected from the three latter groups. At the same time, there was a correlation between poor households and lower ability to access employment, particularly migration work. Within the changing economic context, the employment circumstances for households changed during the project period. Families that might be considered affluent one year, may become a poor family the following year.  This pattern meant that the project approach of defining the poor in three categories became confusing for implementation. Counties that adhered to the project definition faced greater challenges than those which worked more strategically to support economic development for the whole village.

Migration employment. Income from migrant labour makes a substantial contribution to the increase in household income in the project area. During the project period, the income was used to increase living standards, invest in assets, create local employment and provide informal loans, all of which complemented project activities. Yet there are negative impacts of migration. The labour structure of agriculture has greatly changed. Women and elders do most of the labour work. Families are separated and household decision-making is affected. The long term impact of migration on the community is not yet fully recognized.  There is likely to be a high sensitivity to employment changes in relation to external economic forces.

Investment scale versus equity. Project investments were not evenly spread across the targeted villages. Benefits were higher in villages where the investment was sufficient for multiple and synergistic inputs. Yet, there was capture of a greater proportion of benefits by VIGs with greater capability in planning and lobbying for resources. Villages that only accessed funds to assist the main village centre and or where there was a greater focus on one project intervention did not achieve the same level of development.  In some areas, remote village households were missed out by the project. 

Local capacity-building. Management and maintenance of infrastructure is still problematic because in most villages there was no formal maintenance mechanism. Accordingly, the efficiency of the project was reduced by the destruction of infrastructure caused by calamities such as floods and land slides. Apart from the insufficient capacity in local infrastructure management, in broad the project did not implement a strategic capacity-building approach.  Training tended to be adhoc at all levels.  Training and orientation in aspects important for good project implementation such as project management, rural finance and gender approaches was insufficient.

The project design called for support for developing community organizations such as farmers associations and infrastructure maintenance groups.  Such activities did not eventuate as a project strategy. 

Management information system. Insufficient assistance was provided to the PMOs to adjust the management information system at an early stage to generate the required reporting data in relation to the project logical framework. The lack of an effectively functioning management information system contributed to lost opportunities in adapting the project design to the requirements of the project area. With improved information available from the PMOs, the Ministry of Finance and IFAD would have facilitated better knowledge management and management decision-making.

Recommendations

Proceed with a follow-on project. The positive performance of the project demonstrates the value of investment in remote areas of the country. The continuing pockets of need for poverty reduction suggest that there is a valid reason for proceeding with a follow-on project. The design process should retain key elements such as: the focus on targeting of the poorest households, increased participation of women and farmers in village planning processes, an integrated package of interventions, and a project management process that strengthens local governance. The design of a follow-on project should also consider the following recommendations.

Increased flexibility in project design. The pace of change in rural China is very rapid. There are many initiatives that have overlapping impacts in the project area.  It is recommended that there is a more explicit recognition of the changing context in project design and implementation. It may be more appropriate to consider eligibility for progress redesign at each mission rather than delaying until the project mid-term. Similarly, definitions of poverty groups should not be rigid so that the project can respond to changing needs of the households in the project areas. In particular, any credit support should allow for a mix of credit sources, depending on the capacity of the potential borrower at different stages of the project. This could include community development funds and small group lending for the lowest income groups, collective credit mechanisms for cooperative activities, and RCCs for larger enterprise loans. IFAD and the GOC should work together on identifying which aspects of project design must be compliant for the whole project period and which aspects can be subject to change in line with project conditions.

Support to employment generation as a design strategy. It is likely that migration employment will continue to be a major factor in rural areas for the near future, but long term changes may bring other changes and challenges. If a further project is approved, further analysis is needed to understand the characteristics of migrant labour and the attitude of households towards external employment opportunities. This may result in consideration of special credit facilities to support migration, or organizational support for workers rights. It is also worth to consider the impact on agriculture production and community resilience for those residents remaining in the village.

Scale of operations per village. A clearer investment mechanism is required per village such that the greatest economies of scale and optimum short and long term benefits can be achieved. In project design, guidelines for an investment quota could be defined.  The participatory decision making approach of the VDP could be replicated but with greater analysis of feasibility and more stringent approval processes for any proposals that exceed the quota; and additional capacity-building for VIGs which are having difficulty in reaching the quota should be considered.

Greater local mobilisation and capacity-building. More focus should be placed on investing in local organizations, leadership and community activities that will assist in achieving greater levels of results across all impact indicators. Establishing infrastructure operations and maintenance groups would provide greater protection of project investments in physical facilities. Initiating and supporting farmers' producers and marketing organizations would provide an additional stimulus to the local economy. It would also align strongly with the current GOC policy towards strengthening farmers organizations. Strengthening local organizations would require a specific sub-component, with relevant performance indicators and suitably qualified staff.

Management information and analysis. IFAD must work with the GOC to install effective Management Information Systems (MIS) at the commencement of a project. It is critical that data is collected in a way that allows data to be easily checked for accuracy and is linked to project outcomes. Supervision missions require specific guidance on checking compliance with the MIS design and ensuring that the guidelines are followed. The generation of quality data would be a valuable resource for the GOC and IFAD in future to provide deeper insight to strategic considerations such as the attribution of project investments to the economy and social well-being in the project areas, and more broadly, learning regarding the key success factors in poverty reduction in China context.


1/ World Bank Development Indicators, 2006.

2/ Report and recommendation of the President to the Executive Board on a proposed loan to the People's Republic of China for the Qinling Mountain Area Poverty-Alleviation Project.

3/ Loan terms were highly concessional - 40 years, with a grace period of ten years and service charge of 0.75 per cent per annum.

 

LANGUAGES: English

West Guangxi Poverty-Alleviation Project (2010)

China  
December 2010

Completion Evaluation

Introduction

In line with the Evaluation Policy of the International Fund for Agricultural Development (IFAD), the Fund's Office of Evaluation (IOE) has undertaken in 2009 a completion evaluation of the IFAD-financed West Guangxi Poverty-Alleviation Project (WGPAP) in the People's Republic of China. The main purpose of this evaluation was to assess the performance and impact of the project and generate lessons learnt that will guide the Government of China and IFAD in ongoing and future agriculture and rural development programmes in the country.

The basis for the completion evaluation was the self-assessment prepared by the Asia and Pacific Division of IFAD (APR), building upon own insight during project implementation and on the Project Completion Report (PCR). The Desk Review Note, prepared by IOE prior to the main mission, was another other important source of evidence. Mission activities concentrated, among others, on discussions with focus groups and individual households, and drawing contests organized in five primary schools, one in each county visited. These contests aimed at revealing impact domains in terms of poverty impact as perceived by schoolchildren. The mission also prepared 48 scorecards focusing on village implementation groups (VIGs), 20 detailed household interview records, related to the rural finance component, and conducted a ranking exercise of project strengths and weaknesses at the wrap-up meeting in Nanning, which will be referred to in this evaluation report. Thus, the mission attempted to triangulate perceptions of project performance and impact, by taking into account viewpoints of the project's stakeholders, beyond the documentation prepared by the provincial and county Project Management Offices (PMOs) for the evaluation.

With a population of 1.31 billion inhabitants in 2007 and a land area of about 9.6 million square kilometres, China is the most populous country in the world and the third largest in terms of land size.

During the past 25 years, China achieved remarkable economic growth of more than 9 per cent per year and managed to keep inflation in check over the last ten years. In 2005, the country ranked fourth in the world in terms of gross domestic product (GDP) as a result of the economic reforms introduced three decades ago. Because of this sustained rapid economic growth, China is ahead of schedule in achieving the Millennium Development Goals for national poverty reduction. More than 400 million people have moved out of poverty by increasing agricultural productivity and creating productive employment opportunities in the service and manufacturing sectors.

The economy has become more urbanized than before, increasing the urban poor population up to 41.8 per cent in 2004. The distribution of economic benefits among economic and social groups has not been even, as inequalities still persist in three dimensions: i) rural-urban; ii) east-west (geographic); and iii) inland-coastal regions.

The national poverty reduction strategy had a marked focus on mountainous and other marginal areas, and on multi-sector poverty reduction programmes. Poverty alleviation efforts were focused on poorer administrative villages and on reaching out to all natural villages within one administrative village. Participatory approaches and capacity building at community level were increasingly seen as a major leverage factor in the development of marginal areas.

The WGPAP was approved by the IFAD Executive Board in December 2000 and declared effective in March 2002, with an IFAD loan portion of US$30 million and total project costs of US$106 million. The long term goal of the project was to achieve sustainable and equitable poverty reduction in the karst area of the Guangxi Zhuang Autonomous Region (GZAR). The overall objective was to achieve a sustainable increase in productive capacity, both on- and off-farm, and to offer increased access to economic and social resources, including financial services, education, health and social networks in ten of the poorest counties of GZAR. WGPAP was one among five integrated rural development projects that have been launched during the period 1999-2005 in the framework of the 1999 country strategic opportunities paper (COSOP), but the only IFAD-funded project designed in Southwest China during that period. The project included five components, namely: i) Agricultural Development; ii) Financial Services; iii) Social Development; iv) Rural Infrastructure; and v) Project Management. In line with the 1999 COSOP, explicit partnerships were built into the design of WGPAP with the World Food Programme (WFP) and the Rural Credit Cooperative Union (RCCU) network in the project area. While a provincial PMO oversaw project implementation, the bulk of project activities was carried out by county and township PMOs.

Project performance

WGPAP was relevant for the needs of the rural poor and addressed all important dimensions of poverty. Its design was consistent with both national policies and IFAD's strategies, and integrated lessons from other IFAD projects in China. The whole design process was predominantly based on national expertise and can be considered as good practice. Effectiveness was highly satisfactory specifically measured by indicators in the dimensions of food security, income generation, health, education and enterprise start-ups. One of the key ingredients of effectiveness was the high coverage achieved across the ten project counties. Overall efficiency was satisfactory, but the project failed to conclusively prove that the economic rate or return (ERR) at completion was probably higher than the ERR assumed at appraisal. The study commissioned to this effect by the project, while commendable as such, lacked the necessary methodological rigour.

Poverty reduction impacts of WGPAP were in the satisfactory range for all five impact domains. Despite the existence of mainstream poverty-alleviation programs funded by the Government, impact in the project counties and townships could be attributed to the project with a fair degree of certainty, whereby the mission consciously triangulated impact perception from observation points that were additional to the PCR, which was admittedly the main source of evidence. Household income and assets progressed at a significant pace, inferring that the poverty reduction targets have been attained.

In the domains of social capital and empowerment, food security and natural resources and environment, the evaluation mission also perceived significant impacts, derived from capacity building thanks to broad based literacy courses and agricultural training of which women were key beneficiaries. Food security was improved due to substantial productivity increases in cereals and sweeping crop and animal production systems diversification. The evaluation mission noticed, in relation to the impact domain of institutions and policies, remarkable progress in service delivery capacity of county and township authorities as well as of the Rural Credit Cooperative (RCC) network, but also had to recognize that generic institutional capacity at administrative village level remained weak.

Overall sustainability is rated as satisfactory. Two key factors of sustainability are the existence of mainstream government programmes in poverty alleviation - to which WGPAP was complementary - and the seamless integration of the PMO at county and township levels into local government structure. These factors outweigh areas of constrained sustainability, such as the still perceivable subsidy elements in the formation of interest and refinancing rates of the RCC network and the high level of public service delivery in remote areas. In terms of innovation and replication, the project went at length in capturing and documenting innovations, but it received insufficient innovation inputs from IFAD, despite the pronounced innovation drive in the 2005 COSOP.

A salient trait of the WGPAP evaluation is that all involved partners exhibit performance ratings in the satisfactory range. Government and beneficiary performance ranked best, not only for their ability to implement the project effectively and efficiently, but also for the determination of the PMOs at county and provincial level to establish and run a comprehensive monitoring and evaluation (M&E) system. WGPAP would have had the opportunity of transforming this M&E system into a genuine and real time management information system (MIS) if appropriate guidance and support from IFAD had been provided. Despite this shortfall, both IFAD and the United Nations Office for Project Services (UNOPS) performance ratings are in the satisfactory range, marked by continuity and consistency in personnel and concepts that have marked a clear line of business. In fact, WGPAP was seen as a largely problem-free operation by all involved actors.

Findings and Recommendations

Findings

Summarizing the main evaluation findings, a storyline of relative success emerges. The table below underlines this with ratings that are all in the category of satisfactory achievements. Compared to the ratings contained in the Desk Review Note and to those made in the self-assessment of APR, only minor variations emerge. On the other hand, WGPAP ratings fare significantly better than the ones of the Annual Report on Results and Impact of IFAD Operations (ARRI) 2008.

Evaluation rating summary

Evaluation Criteria

Ratings Completion Evaluation

A. Project performance  
Relevance 5
Effectiveness 6
Efficiency 5

Overall project performance 5
B. Rural poverty impact  
Household income and assets 6
Social capital and empowerment 5
Food security and agricultural productivity 5
Natural resources and the environment 5
Institutions and policies 5

Overall rural poverty impact 5
C. Overarching factors  
Sustainability 5
Innovation, replication and scaling up 5

D. Performance of partners  
IFAD 5
WFP 5
UNOPS 4
Government 5
E. Overall project achievement 5

 

 Although the rating summary is a comprehensive and well tested analytical framework for the assessment of project success or failure, it falls short of convincingly explaining its underlying contributing factors. In a simplified manner, project success may be seen as a function of quality at entry, implementation capacity and conducive environment. Quality at entry of WGPAP was marked by: (i) an early and unwavering determination of and agreement on key fundamentals of the project, by and between the main partners, (ii) solid partnerships established along proven patterns, (iii) strong participation and ownership of the national partners during design, (iv) participatory mechanisms involving the target population on a broad basis, and (v) a component mix and cost sharing mode that reflected perceived needs as well as comparative advantages. WGPAP's implementation capacity was driven by: (i) the experience and credibility of the provincial PMO to provide decisive guidance, (ii) the presence of committed and qualified human resources, sufficient in numbers and able to transform guidance into action, and (iii) the continuity of these human resources. The fact that poverty impact, sustainability and innovation were satisfactory stemmed from such implementation capacity, on one hand, and from a conducive environment, on the other hand. The project environment was conducive because: (i) project implementation was supported by a set of government policies that had a clear rural poverty reduction focus and that emerged when the WGPAP was designed, (ii) local government structure was capable, possibly by design and certainly in reality, to practically integrate the project as a mainstream undertaking, and (iii) a stable political environment and a booming economy during project.

Recommendations

The evaluation mission makes four recommendations as detailed below. The first refers to the design of future IFAD projects in China, the second to key aspects of institutional partnerships, in particular regarding rural financial services, while the third recommendation points to innovation and IFAD's role of its promotion. The fourth recommendation deals with M&E and MIS. It is understood that the government of GZAR intends to propose a second project to IFAD that would be based on lessons learnt form the WGPAP.

Recommendation 1: Design of future IFAD-funded projects in China. The design of future IFAD-funded rural development projects in China should continue to be responsive to the multidimensional character of poverty and the growing expertise of national stakeholders in this kind of operations, in particular: 

  • Targeting of project townships should continue to be based on factual indicators. In projects focused on service provision, all villages within one township should be included in the project, and natural villages more clearly identified. Likewise, a transparent assessment of the special needs of the various ethnic minority groups is critical.
  • The design process should replicate WGPAP's good practice of strong involvement of local expertise at all levels, including from the anticipated target population.
  • In settings where food-for-training and food-for-work is deemed essential for broad participation and coverage, a second-generation solution regarding food aid from domestic sources should be devised with the Chinese authorities, in order to replace WFP food aid.

Recommendation 2: Institutional partnerships. Project partnerships should extend over the boundaries of provincial and local government in the strict sense, e.g.:

  • VIGs or analogous bodies at village level should not only play a role for planning and monitoring purposes, but should be strengthened in their core management capacity, with special regard to service delivery functions and equitable gender representation in positions of responsibility.
  • Partnership with provincial RCC networks should be put on a completely new footing, by taking into account the ongoing reform and by agreeing on a set of information to be shared that is conducive to a real-time assessment of loan portfolios and banking performance indicators in general.
  • Key areas of policy dialogue should be determined systematically and given due and documented attention over the project implementation period.

Recommendation 3: Role of innovation in IFAD-funded projects. Building on the orientations of the 2005 COSOP, new IFAD-funded projects should put significantly more emphasis on innovation and its promotion, in particular:

  • Opportunities for innovations should be screened systematically from formulation stage onwards and be periodically reassessed.
  • Particular attention should be devoted to the identification of local sources of innovation and their mobilization during project implementation.
  • Innovations that exist or emerge in the global context should be made available to IFAD-funded projects more pro-actively, possibly also via IFAD grants.
  • Priority areas should encompass innovations that respond to global challenges, such as food security, soil fertility, alternative energies and climate change.

Recommendation 4: From M&E to MIS. New IFAD-funded projects should take stock from the accumulated experience of the WGPAP and actively foster a quantum leap in the state-of-the-art of M&E and MIS, with the aim to make advanced M&E and MIS methods a mainstream feature in China. This would include the following:

  • The generic requirements of an overarching, yet concise, M&E system for multidimensional poverty-alleviation projects in marginal rural areas should be defined, best in a joint effort between national expertise and IFAD resources.
  • The basic architecture of a real-time MIS, based upon such M&E requirements, should be designed in a way that it can be used for a wide array of IFAD-funded projects, in China and possibly also elsewhere. Again, a partnership between national expertise and IFAD would be advisable.
  • The above recommendations would give a particularly concrete meaning to the development of local evaluation capacity, an initiative that IFAD has decided to support in China. 

 

 

LANGUAGES: English

Raymah Area Development Project (2010)

Yemen  
December 2010

Introduction

Objectives. As agreed with the Executive Board, the IFAD Office of Evaluation (IOE) undertook an evaluation of the IFAD-financed Raymah Area Development Project (RADP) in the Republic of Yemen in 2009.  The main objectives of the evaluation were to: (i) assess the results of the RADP; and (ii) generate a series of findings and recommendations that could inform future agriculture and rural development projects in the country. The evaluation was undertaken in line with the provisions contained in IFAD's Evaluation Manual.

Project background. The RADP was approved by the Executive Board in December 1997. It became effective in July 1998 and closed in December 1997. The total project cost was US$17 million, out of which IFAD provided a loan of USD 1.2.1 million, the Government and beneficiaries provided USD 3.9 million, and USD 1 million was to be provided in grant by a cofinancier to be identified. The cooperating institution was the United Nations Office for Project Services (UNOPS) responsible for project supervision and loan administration.

The project objectives were: (1) to improve living conditions in Raymah through the provision of sustainable rural infrastructure and services and consolidating strong community organizations to articulate the community aspirations; and (2) to increase rural incomes on a sustainable basis by enhancing the productivity of small holdings. The project had four components: rural infrastructure, community development, agriculture development and project management.

Project performance

Relevance

RADP objectives were aligned with the Government's strategy as captured in its five-year development plan 1996-2000, which included an emphasis on promoting development in Raymah. Objectives were also congruent with the 1997 COSOP, which identified the needs of poor rural people in terms of, among others, rural infrastructure (water supply, and rural roads for improved access), and the development of technologies to enhance productivity and food security.  At the same time, however, the project took a multisector approach by including in its design the financing of many activities in a diverse range of subsectors, adding to the project's implementation and coordination complexities. The implications of the prevailing sociocultural context, insufficient community participation experience, and the Government's institutional capacities and outreach were not adequately taken into account at entry, which led to challenges during implementation. Finally, the choice of the Cooperative and Credit Agriculture Bank (CACB) is questionable given the bank's limited experience in group lending at the time. All in all, the relevance of the RADP is assessed as moderately satisfactory.

Effectiveness 

The effectiveness of the RADP is assessed in relation to its two main objectives.

Improving living conditions in Raymah through the provision of sustainable rural infrastructure and services and the setting up of strong community organizations able to express community demands and aspirations. The provision of small-scale infrastructure (such as small dams, feeder roads, drinking water supply) contributed to improving living conditions in the project area. Poor rural communities have better access to safe drinking water and some access to market opportunities. Women's workload has been reduced as they no longer need to fetch drinking water. Feeder roads contributed to lowering transport costs (e.g. the costs of transporting a 50 kg sack of wheat in Al-Malqah was reduced from 200 to 75 Yemeni rials (YER). 1

However, the larger and much more cost-intensive rural infrastructure (e.g. the construction of large dams and the main trunk road) was not properly carried out, and therefore its contribution to effectiveness has been below expectation at design and generally unsatisfactory. Similarly, an important project activity was to make available sustainable rural finance, through which the target group could enhance their on- and off-farm income-generating activities and derive better incomes. However, on the whole, the rural finance activities under the project did not work.

Although the project made efforts to mobilize and organize communities, there were only a few groups and associations still functioning at the time of this evaluation and the involvement of women in them was low overall. There were a variety of reasons for this, including (i) insufficient capacity-building provided to community groups/associations; (ii) community groups/associations formed primarily to access project resources, rather than as lasting instruments for rural poverty reduction; and (iii) the Government's decision not to recruit an international NGO with expertise in participatory approaches and community mobilization, as originally envisaged at appraisal.

Increasing rural incomes on a sustainable basis by improving the productivity of smallholdings. There were improvements in productivity, for example as a result of the small dams for irrigation of vegetable and fruit plots, improved animal husbandry practices, and investments in small-scale irrigation infrastructure. Livestock farmers benefited from an average increase in milk productivity from 2.6 to 3.9 litres. Farm revenue increased by YER 50 per hectare for sorghum and YER 200,000 for corn in irrigated areas. However, these have not translated in sustainable improvements in incomes especially among poor rural people. A main reason for this was inadequate access to agroprocessing facilities and marketing opportunities. In conclusion, the evaluation's assessment of RADP effectiveness is moderately unsatisfactory. It is important to bear in mind that this overall judgement on effectiveness incorporates significant variation among project activities.

Efficiency

On the positive side, the time lag between RADP loan approval and loan effectiveness was seven months, which is better than the average time taken for IFAD-funded projects in Yemen. However, the RADP took 9.5 years for implementation, as compared with an average of 6.8 years taken for all IFAD-funded closed projects in Yemen. The longer implementation duration of the RADP caused additional unforeseen administrative and transactions costs for IFAD and the Government alike, thereby reducing efficiency. The complex and lengthy procedures for release of funds within the Government was a factor affecting the efficiency of implementation. CACB faced high transaction costs in loan approval and management processes. The Beit Al-Faqih to Al-Hadia trunk road was to be constructed within 18 months of loan effectiveness, but protracted delays and poor management meant that the works had not completed even at project completion. There are many other examples that illustrate poor efficiency, including the resources spent on big dams, which are largely not operational. Overall, RADP efficiency is rated as moderately unsatisfactory.

Rural poverty impact

The construction of the main trunk road, in spite of the shortcoming reported earlier, and the feeder roads have provided some access to markets and employment opportunities. The feeder roads are benefiting around 13,000 people in remote areas of Raymah. Women's incomes in some cases have also improved as a result of training in income-generating activities (sewing and embroidery, food making), but the most important aspect has been better access to safe drinking and irrigation water. In terms of human and social capital and empowerment, the RADP had a positive impact on health and human security, especially through the water reservoirs and supply systems. As a result, life risks have been reduced and women's workloads have been reduced. It is estimated that households save an average of 180 hours per year in fetching water.

The impact on creating strong community organizations and individual empowerment has been rather limited, which has constrained communities' ability to advocate with local or national authorities for better services and development activities. Some impact was achieved on agriculture productivity and food security as documented in various self-evaluation reports, which, however, was not entirely evident at the time of evaluation.  The least favourable impact is in the area of natural resources management and environment. Raymah is characterized by great environmental fragility and erosion potential. The project did not take sufficiently into account the area's topography and agroecological conditions, especially in the building of large dams. For example, the large dams were constructed without a slope stabilization programme through plantations or terrace rehabilitation in the upper reaches of the dams. In fact, the RADP lacked a systematic and rigorous assessment of environmental impact. The mountainous segments of the trunk and feeder roads constructed by the RADP are exposed to the risk of landslides, as construction works were carried out with minimal protection of mountain sides and downhill agricultural land. In summary, the rural poverty impact of the RADP is considered moderately unsatisfactory.

Sustainability and innovation

Sustainability. The evaluation found the prospects for sustainability of some of the small infrastructure to be promising (e.g. open water reservoirs, drinking water schemes, feeder roads). Better sustainability is expected in those cases where investments were identified and developed with the participation of beneficiaries.

However, the sustainability of dams (especially the larger ones), the trunk road, the electricity scheme and the mechanized drinking water supply system is weak. Major financial resources were invested for these activities from the RADP. For large infrastructure, the lack of sustainability stems from inadequate technical design, low maintenance capacity and limited allocation of counterpart funds for recurrent costs. As already mentioned, the evaluation also concluded that the sustainability of community organizations and associations was weak, and noted that the provision of research and extension services had already ceased during project implementation. The rural financial services system is not considered sustainable given the extremely low repayment rates, and the women's centres constructed are mostly no longer in use. No exit strategy was developed during implementation or close to project completion to define an approach for sustainability. In conclusion, the sustainability of benefits is therefore considered unsatisfactory.

Innovation and scaling up. The promotion of community participation and empowerment and the development of social capital were the most innovative aspects of the RADP. The extension system based on a network of extension intermediaries was also innovative, as was the development of dedicated centres and the hiring of women as rural extension officers as a means to encourage their involvement in development activities. Efforts were made to innovate through research in new agriculture technologies, for example, by introducing biogas plants in households with large livestock herds.

Although the design included innovative elements, a large number of these initiatives could not be successfully implemented. For instance, RADP design severely underestimated the time and effort that would be required to change mindsets in government and among the beneficiaries to promote participatory approaches, and undertake activities that would contribute to developing gender equality and women's empowerment. In this regard, as mentioned earlier, the non-hiring of an international NGO to support social capital-building and empowerment did not help either. The adoption rates of new technology were low, as adaptive research activities did not always build on community priorities, and there was limited knowledge and understanding among beneficiaries of the farming systems promoted. Finally, the project did not have an approach for scaling up, even though it left an impression among those who were involved in its implementation of having attempted to introduce approaches that were not common in the country at the time. Innovation and scaling up are rated moderately unsatisfactory.

Performance of partners

IFAD. One of IFAD's merits was to assist the Government in undertaking a project in a remote and barely accessible area that had not previously been the focus of any major development intervention. The RADP contributed to increasing government policymakers' and donors' attention to Raymah. During implementation, Raymah was granted a full governorate status, which has far-reaching implications for its development in the future. On the other hand, IFAD shares responsibility for the design of a project that proved unsuitable to local conditions and capacities. The troubled project implementation was repeatedly emphasized in the annual supervision mission reports and the mid-term review. The fact that IFAD Management itself, in the context of its annual internal portfolio review process, classified the RADP as "at risk" for 8 years in a 9.5 year period of implementation is further confirmation of the severe problems that the project faced during implementation.

IFAD on its part did not take a firmer position with the Government to ensure the implementation of supervision mission and mid-term review recommendations, which could have contributed to better effectiveness in the end. There is also evidence that the IFAD Country Programme Managers for Yemen at the time only visited the project on three occasions throughout its implementation, even though supervision and implementation support was outsourced to UNOPS, in accordance with the IFAD operating model of the time. That said, the evaluation is aware that IFAD Management in general, in the recent past, is increasingly paying attention to portfolio management issues and there are examples of projects similar to the RADP that have been cancelled prematurely before their intended completion date. The situation in terms of dialogue and communication with the Government is remarkably different now. The current IFAD Country Programme Manager is responsible for direct supervision and implementation support of all ongoing projects in the portfolio, and maintains a close, constructive and appreciated dialogue with partners in-country. All in all, however, the performance of IFAD in the RADP is considered unsatisfactory by the evaluation.

Government of Yemen. The Government should be commended for taking the important initiative for transforming Raymah into a governorate in 2004, during the implementation of the RADP. The Government has shown maturity in accepting the findings of this evaluation constructively, and in being self-critical of past actions, which they are committed to avoiding in the future. However, in general, the poor design of the project is a responsibility the Government shares with IFAD. The fact that the RADP was financed as a loan should have served to instil in the Government a greater sense of ownership and responsibility, and motivated it to ensure better management of activities and use of resources. The Government also had a key role in nominating the first project director, which was done on a non-competitive basis, leading to the designation of an individual whose performance was inadequate. Delayed provision of counterpart funds and payment of salaries affected staff performance and motivation and hence RADP effectiveness. The performance of the Agricultural Research and Extension Authority (AREA) was inadequate in a number of aspects, especially its undertaking of research on activities that were not of priority to poor rural people. Despite the difficulties encountered during implementation, as with IFAD, the Government also did not take stern measures to monitor implementation (e.g. the project steering committee did not meet as regularly as required) and bring the project back on track. CACB, which was fully a government undertaking during implementation of the RADP, on its part did not make special efforts to ensure that loans went to the IFAD target group, and most of its portfolio was non-performing.1 In conclusion, the Government's performance overall is rated as unsatisfactory.

UNOPS. The evaluation found that UNOPS organized missions regularly, summarizing implementation progress adequately. On a less positive note, the evaluation found that supervision missions were limited in terms of their technical depth and analysis, which could be to some extent attributed to the restricted level of resources made available by IFAD for the function. UNOPS did not rigorously follow up on the implementation of the recommendations of the mid-term review. In sum, the performance of UNOPS2 was also unsatisfactory, even though it is of relative unimportance at this stage, given that IFAD is now undertaking direct supervision and implementation support of the Yemen portfolio.

Conclusions and recommendations

Conclusions

The RADP was implemented in a geographic region marked by challenging characteristics, especially in terms of its topography and its agroecological, socio-political and cultural context. The region had largely been neglected by the Government and other donors, and the evaluation therefore commends the Fund for its initiative to finance the RADP in 1997. The evaluation found that the project played an important role in opening up the Raymah region to more comprehensive development investment activities by the Government, and in bringing critical services to the area. Moreover, the RADP can be credited, at least to some extent, for the transformation of Raymah into a full-fledged governorate in 2004, thereby increasing its visibility among policymakers, donors and development practitioners.

In addition to the above, the project made some useful contributions to rural poverty reduction in the region, even though more could have been achieved. The small- scale infrastructure developed by the project (e.g. small dams, water harvesting reservoirs, community drinking water supply schemes, and feeder roads) provide important benefits to small farmers.

The large dams constructed have not proved to be successful for a variety of reasons including technical, operational and maintenance limitations. The Beit Al-Faqih to Al-Hadia trunk road, which is extremely important for the region, has not yet been fully constructed as envisaged at appraisal. Its construction and quality was affected by the difficult relationship between the project management unit and the selected contractor. The capacity and performance of the contractor was also inadequate. Moreover, as for large dams and the water supply system in general, the construction of the trunk road increased attention to several natural resources management and environmental issues that were not adequately addressed by the RADP and remain a priority. For example, in constructing roads through mountainous terrain, the project did not consider protection of downhill agricultural land and landslides of material removed to make the roads.

On another issue, the only electric scheme constructed, in Kusmah, is no longer functioning mainly because of the high costs for its operations and maintenance, and a weak system for collecting fees. This is unfortunate, as the provision of electricity had contributed to improving livelihoods among poor rural people, for example, in terms of the mechanization of the water supply system and for literacy promotion activities.

The project was instrumental in furthering understanding of the concept of participatory planning in Yemen, and encouraging community involvement in resource allocation and the implementation of development activities. However, project design overestimated government capacity at the time for community mobilization and participation-building, especially among beneficiaries who had not previously been exposed to such approaches. As such, the sustainability of community organizations and associations is a cause for concern. The expected support of an international NGO for developing social capital and mobilizing beneficiaries did not materialize, which could have proved useful in achieving this specific RADP objective. Finally, the project did pay some attention to gender issues, for which an international adviser was hired for two years in 2002, even though in the end little progress was made in promoting gender equality and women's empowerment.

The RADP supported the development of the agriculture extension system, even though only 43 out of the 150 extension intermediaries foreseen were selected and trained. Salaries and allowances for extension staff were either not provided or often provided late, affecting staff performance and morale. This led to the collapse of the extension system in 2005. A number of veterinary officers trained under the project eventually set themselves up as private service providers operating livestock pharmacies, which is a useful achievement. Research was conducted through AREA on a number of new crop varieties, even though some crops (e.g. high-yielding varieties of sorghum and peanuts) could not be adopted for various reasons: farmers lacked the requisite knowledge to do so; not all of the new technologies responded to community needs and priorities; and access to markets was limited. Four of the five nurseries set up under the project to supply farmers with horticulture plants and forestry crops were no longer operating at the time of the evaluation, due to lack of water and the unwillingness of private entrepreneurs to take over.

With regard to rural financial services, even though close to 700 individual loans were provided for a variety of on- and off-farm activities, the repayment rates were very low, jeopardizing the sustainability of the rural financial activities. Several of the loans benefited relatively more influential individuals and only a small portion of funds were provided to women.

Project management was troublesome, as recognized by the Government. For example, the first project director (who remained in the position for four years) was not selected through a competitive process and was recognized not to have the required skills and experience to manage such an operation. Likewise, the Government acknowledged that initially locating the project management unit in Sana'a was an inappropriate decision. Government monitoring efforts were limited and well below the levels required given the challenges faced in implementing the RADP.

IFAD, on its side, did not follow up promptly to redress key constraints during implementation, despite having itself classified the RADP as a "project at risk" for 8 out of the 9.5 years of the project's implementation. The evaluation considers that allowing a project with serious problems such as the RADP to continue underperforming year after year was a significant shortcoming on the Fund's part at the time. However, the recent decisions to undertake direct supervision and implementation support, strengthen portfolio monitoring and management in general, consolidate country presence, and assign a new IFAD Country Programme Manager for Yemen in 2008 are steps in the right direction, which are collectively expected to improve the overall partnership between IFAD and the Government in the future.

Recommendations

Allocate required resources for ensuring adequate understanding of country context. In the design of future projects, it is important that objectives and activities are commensurate with the institutional, social and economic capabilities of the country. In this regard, sufficient administrative resources should be allocated by Management to ensure that adequate analytic work can be undertaken during project design in Yemen, which would enable a thorough understanding of the country's policy and institutional environment. This is especially important in light of the weak institutional and policy framework in the country.

Pay attention to natural resources management and environmental issues. Given the concerns for depletion of underground water resources in Yemen, the environmental impact implications of IFAD-financed operations and the proposed coping strategies should be clearly articulated in design. Construction of large infrastructure projects should systematically include comprehensive and rigorous environmental and social impact assessments, before activities are commissioned. This would be consistent with the Fund's environmental and social assessment procedures, which were in fact followed in the design of the most recent operation in Yemen. 3

Increase involvement of the private sector in agriculture research and extension. In the light of the past experiences with and the capacities of the Government's institute for agricultural research and extension, it is recommended that opportunities be further developed to involve producers' associations and other value chain actors more actively in the provision of such services to poor rural communities. This would be in line with the strategy adopted in the more recent IFAD-funded portfolio in Yemen of a wider engagement of private-sector entities.

Build on community priorities. It is essential that future adaptive agriculture research and extension activities be adequately tailored to ensure that farmers receive technologies and advice that allow them to address their priorities for agriculture and food security.

Focus on development results. Measures to ensure a continued focus on development results are many, but the main ones could be ensuring that: project objectives are achievable and measurable; project supervision pays attention to results in addition to achievement of physical and financial targets; the implementation of supervision recommendations are monitored and promptly followed up; project-level monitoring and evaluation systems are well resourced; and baseline surveys are undertaken during design or at the outset of implementation.

Improve sustainability. In the context of small-scale infrastructure development activities, attention should be given to building management capabilities among beneficiaries and their groups for operations and maintenance. Exit strategies should be developed early on in the process, well ahead of project closure.

Complete the Beit Al-Faqih to Al-Hadia road. The Government should consider including in its own development plans the completion of the road from Beit Al-Faqih to Al-Hadia, which is critical to further improvements in connectivity, access to markets and communication.


1/ US$1 = YER 215.6

2/ This is also confirmed in a report on microfinance in Yemen issued in June 2005 by the Consultative Group to Assist the Poor.

3/ The Economic Opportunities Programme, approved by the Board in April 2010.

 

 

LANGUAGES: English

Rural Development Programme for Las Verapaces (2009)

Guatemala  
December 2009

Interim Evaluation

Executive summary

 

Background

In December 2007, the Executive Board of the International Fund for Agricultural Development (IFAD) requested the Office of Evaluation (OE) to undertake an interim evaluation of the Rural Development Programme for Las Verapaces (PRODEVER), implemented in Guatemala. Pursuant to IFAD's Evaluation Policy, an interim evaluation is obligatory before initiating a second phase or starting up a similar project in the same region.

Evaluation objectives, methodology and procedure. The main purpose of the evaluation is to assess PRODEVER's performance and impact, seeking at the same time to generate findings and recommendations that will serve as guidelines for IFAD and the Government of Guatemala when carrying out a similar projects in the region.

The evaluation was conducted in accordance with the evaluation manual of IFAD's Office of Evaluation. A preparatory mission took place from 2 to 12 April 2008. The main evaluation mission to the field remained in the country from 30 April to 23 May of the same year. Upon completion of the field mission, an aide-memoire was presented in Guatemala City highlighting the main preliminary findings and key issues encountered during the main mission.

At the end of January 2008, there was a change in government in Guatemala. One major consequence for the programme has been that, in late February 2008, the National Fund for Peace (FONAPAZ), the implementing agency for PRODEVER, replaced much of the programme team, limiting the availability and processing of programme information.

In view of the limited information available on the programme, the evaluation mission decided to conduct a survey with a randomly selected sample of 33 PRODEVER-supported organizations, to obtain data on the interventions and on the programme's effects and impact. This information was triangulated with programme data and interviews with actors involved in implementing PRODEVER.

Country context. Guatemala, with 13 million people, is the most populated country in Central America. It is a middle-income country and ranks 118th of 177 countries in the Human Development Index. Guatemala is characterized by a high degree of income inequality, with a Gini coefficient of 0.57, a level exceeded only by Brazil in Latin America.

The country's recent history has been characterized by recovery, since the Peace Agreements were signed at the end of 1996 after decades of civil war. The upper classes and the State excluded the majority of the population, particularly indigenous people, from public services, and enormous human rights violations were committed. Over the past decade, the State has had virtually no presence outside the country's capital, and public structures at the municipal level have had to be built from the ground up.

Tax collection by the State of Guatemala is extremely low at all levels (including municipal), at less than 10 per cent of gross domestic product until the end of the 1990s and somewhat higher in recent years.

Agriculture sector and land tenure. Agriculture contributes 22 per cent of GDP and employs one third of the economically active population. The sector is oriented mainly towards export crops (coffee, sugar, cardamom, cacao, banana, fruit and vegetables) and staple crops (maize, beans, rice and vegetables).

Rural development. Conflicts in the country's development have been concentrated mainly in rural areas, owing to the marginalization of indigenous people and violations of their rights. This is apparent, inter alia, in the land tenure situation and poverty levels, as outlined below. Rural development occupied a central position in the Peace Agreements signed in late 1996. The last four governments have sought concepts and instruments to respond to these challenges, and IFAD has been one of their chief international counterparts in this regard.

In Guatemala, inequality of land tenure and access is high and the question of land ownership is one of the most conflictive issues for rural development. Census data show that 2.5 per cent of the country's farms average more than 200 hectares (ha) and occupy most (65 per cent of the country's arable land, whereas 88 per cent average 1.5 ha and occupy just 16 per cent of the country's arable land. Accordingly, close to 90 per cent of farms have less than 7 ha of land, and that about 40 per cent of the active rural population are landless.

Rural poverty. According to 2006 figures, more than half the population live in poverty, and 15.2 per cent live in conditions of extreme poverty. Rural areas post the highest rates, with 72 per cent poor and 24.4 per cent extremely poor. Ethnic disparities are exacerbated by the fact that 75 per cent of the poor are indigenous, most of them living in rural areas. Infant malnutrition is of concern, at an estimated 49 per cent of children between 2 and 60 months of age, compared to 22.2 per cent in the Central American region as a whole. The difference in the prevalence of malnutrition between indigenous and non-indigenous children is striking, at 69.5 per cent and 35.7 per cent, respectively, although the rate is high even for non-indigenous children.

The programme. PRODEVER operated in parts of the departments of Alta Verapaz and Baja Verapaz, which feature different climatic and agro-ecological conditions. The programme's target population included 53,700 rural families living below the poverty line, of which 16,000 were direct beneficiaries. The development objective was to reduce poverty in rural communities living in fragile ecosystems in the poorest departments of Las Verapaces, which bore the brunt of the civil war. The main objective was to support members of local communities in restoring the social fabric and developing a sustainable system of production.

Six specific programme objectives were set to promote issues relating to income, organization, infrastructure, financial services, gender, and natural resources and integrating communities into local and national economies. In order to achieve its objectives, the programme had five components: (i) building local capacity, particularly in connection with associations; (ii) sustainable productive development, with three subcomponents: (a) support for agricultural production; (b) sustainable natural resource management; and (c) support for commercialization and rural microenterprises; (iii) rural financial services, with two subcomponents: (a) institutional strengthening of financial service providers; and (b) credit funds; (iv) socio-economic investments, with two subcomponents: (a) community interest investments; and (b) rural roads; and (v) programme management and coordination.
The programme was designed in 1999 with a total cost of USD 26 million. Loans from IFAD for USD 15 million and from the Organization of the Petroleum Exporting Countries (OPEC) for USD 5 million were ratified by the Guatemalan Congress in December 2000 and declared effective in September 2001. The Government has contributed USD 3,7 million and the beneficiaries an estimated USD 2,3 million. The programme began operations in March 2002 with a start-up workshop. As a result of the prolonged FONAPAZ approval process, the first projects commenced in 2003 after a long lapse following programme design, during which the peace process evolved in the country.

PRODEVER was designed and approved under the Flexible Financing Mechanism (FFM) for a duration of 10 years, and has been implemented in two cycles. The first cycle, planned for completion at the end of 2005, was extended for one year to the end of September 2006. A mid-term review conducted in June 2006, led by Jorge Piña Puig, agricultural engineer, found satisfactory compliance with most of the critical triggers to proceed with a possible second cicle of the programme and allocate reserved funding. The mid-term review led to a series of changes in the programme, from the logical framework to a rescheduling of the loans. By mid-2008, the OPEC loan was fully used and the IFAD loan 70 per cent disbursed. The programme is currently in its second cycle, which is scheduled to run until September 2011. However, PRODEVER is expected to close in 2009.

Performance and results

Implementation results

Strategy pursued. When activities began, programme management had to prioritize among the multiple proposed lines of activity. Communities were grouped according to a set of characteristics extracted from a baseline study. PRODEVER sought to operate in ways that would produce tangible results in the short and medium term. Based on the experience of other development projects in the area, PRODEVER initiated its support by building the capacity of community organizations to formulate and design projects, with preference to those generating income in the short and medium term. In addition, to obtain buy-in from beneficiary communities and organizations, the programme responded to the main request made by many of them: to build access roads. The project portfolio in the first two years reflected this approach. At the same time, the programme focused on identifying traditional crops with the greatest commercial potential, such as coffee, cardamom, cacao, banana and citrus fruit, and began to support production and commercialization through producers' associations.

With respect to building local capacity, training modules were imparted to community representatives in 91 communities to enhance their capacity to formulate demands and manage different types of projects potentially funded by different donors. In addition, to promote income-generating activities, the programme identified producers' associations and strengthened them in management and business issues. Two thirds of the organizations supported were already in existence when PRODEVER began, and another third were created with programme support. To measure progress made by the organizations supported, the programme commissioned a consolidation rating study in 2007. Among the 158 organizations evaluated, two were classified as self-managed, 33 consolidated, 90 in the consolidation process and 24 incipient.

The programme promoted sustainable productive development through 568 projects in 297 communities. The greatest success was obtained on traditional commercial crops (cacao, coffee and cardamom) by providing technical assistance and introducing new technologies to manage these crops, as well as by investing in processing and storage and providing support for commercialization. In addition, the programme opted to select other production lines to diversify traditional production systems. To this end, crops are being diversified to include fruit with good profit potential. Efforts were also undertaken for a broad-based improvement in food security, by facilitating the introduction of backyard livestock in 261 communities and supporting various vegetable crops in 44 communities. In addition, support was provided through PRODEVER for the development of agricultural and non-agricultural microenterprises, focusing on business management training and value chains. A total of 56 microenterprises were supported, with participation by 1,892 families.

The execution of sustainable natural resource management was low (39 per cent of resources provided), mainly owing to a lack of interest among community members in reforestation or family forest management. On the other hand, the introduction of improved stoves in 418 households brought substantial savings in firewood and better cooking conditions (shorter cooking times, more erect position, less smoke) and has been much appreciated by the beneficiaries.

PRODEVER has facilitated the construction or rehabilitation of 171,416 km of rural roads in 57 different projects, supporting the creation of road management committees in communities and equipping them with basic tools to this end.

The financial services component has not been implemented as designed (supply was to be reinforced), and has been realized only in part on the demand side by supporting 30 producers' organizations that are further advanced in managing their own funds. Although the programme commissioned two diagnostic studies (in 2004/2005 and 2006/2007), its management did not find the resulting recommendations persuasive given recent failures by other international initiatives in the area of financial services.

Gender has been a cross-cutting issue throughout all the programme activities. Design began with an analysis of the situation of women who are discriminated against and at a clear disadvantage compared to men. The programme performed a sound analysis to promote greater inclusion of women in organizations and to generate conditions of equality through economic development. The design took into account the experiences of other IFAD programmes implemented previously in Guatemala, demonstrating a high degree of consistency with the policy guidelines of both the Government and IFAD.

Evolution. The programme evolved in response to challenges encountered during implementation and remained in line with the Government's policies. Monolingualism and illiteracy among indigenous communities represented formidable obstacles and necessitated inclusion of a literacy campaign. In terms of food security, in 2005 the Government established the Food Security and Nutrition Secretariat. In line with this policy, PRODEVER strengthened its projects in this area by introducing reproductive health. Another activity line that evolved was support for non-agricultural rural microenterprises. Given the lack of success in promoting this area, the programme complemented it with a project to provide vocational training for young people.

Relevance.The programme design is aligned with the priorities and policies of both the Government and IFAD. PRODEVER attempted to address the many needs of indigenous peoples in Las Verapaces, and to meet the challenges of encouraging sustainable income-generating activities. Based on a participatory diagnostic, a complex programme was formulated that was also flexible, leaving room for learning and adaptation during implementation. Six different target groups were identified without sufficiently conceptualizing the difference in measures to address target groups possessing land and other means of production and target groups living in extreme poverty with no such resources. From the point of view of implementation, the programme was expected simultaneously to address an excessively broad spectrum of issues, all of them important, but which together were enough to overtax the implementation capacity of any programme team.

The results obtained on local capacity-building, supporting traditional commercial crops and building road infrastructure have confirmed the relevance of these areas of intervention to poverty reduction. The same cannot be said of food security, sustainable natural resource management and the development of a supply of financial services in the programme area, where implementation difficulties placed the relevance of design in question. In the case of food security, the introduction of backyard animals and vegetables for family consumption was done in haste, without diagnostics, training or proper follow-up. When the time came to set goals for natural resource management, insufficient consideration was given to (dis)incentives for target groups to invest efforts in this area. With respect to access to financing for the target groups, the programme intervention was limited to demand (training of associations), where it was relevant. On the supply side, the situation did not lend itself to the kind of intervention included in the programme design.

The effectiveness of the programme in achieving its objectives has varied. Two of the six target groups – those possessing land – achieved a marked increase in income through improvements in production and commercialization (in particular of coffee, cardamom, mandarin and banana). The programme also strengthened the capacities of many community organizations supported, with a marked increase in women's participation. In addition, the construction and upgrading of road and agro-industrial infrastructure has been crucial to achieving the objective of bringing small farmers into the economic mainstream. On the other hand, the programme has not been effective in preserving the natural resource base, promoting non-agricultural income-generating activities or promoting greater food security. The latter area requires more time to show the results of training and small investments made under the programme. With respect to improving access to rural financial services, the lack of implementation on the supply side did not limit the programme's success in economic development, but the lack of local microfinance alternatives may have contributed to the limited effectiveness of measures intended to increase food security.

Efficiency. Overall, the ratio of resources invested to results achieved has not been satisfactory owing to dispersion over too many (sub)components, implementing agency approval procedures poorly adapted to implementing multiple lines of activity, and the replacement of all programme team members in March 2008. During implementation, administrative expenses increased from 9 per cent of the initial budget to 21.4 per cent while, following a delay in start-up, only a small fraction of the amount provided for M&E in the original budget was used.

Impact. PRODEVER has made a noticeable contribution to the development of Las Verapaces in terms of communal social organization and productive development, as well as road infrastructure, with a consequent reduction in poverty.

  • Social capital has been strengthened by giving community organizations new knowledge on organizational operations and self-management, and by broadening the local outlook towards communal and regional development. In 2007, 35 organizations reached a certain degree of consolidation, most of them operating in commercial crops. Most of the 158 organizations reached (90) are now in the process of consolidation, and 24 are still at an incipient stage.
  • Arrangements to improve the production and handling of cardamom, coffee, cacao, citrus fruit and banana; to facilitate collection and drying by producers' associations; and to improve sales by bypassing one or two intermediation levels, have enabled 27 producers' associations to benefit from higher global and national prices since 2005; and another 25 cacao producers' organizations are in the process of doing so (the international price for cacao did not fall in 2008). As a result, 250 families have crossed the poverty line by increasing their incomes; the same number had already done so. Given the situation at programme start-up, this represents a real impact that could be enhanced provided the new programme team completes the work begun by the previous team.
  • The programme has made major efforts on behalf of the most vulnerable target groups by promoting training projects, supporting community investments and promoting educational projects in literacy, reproductive health, environmental issues and food security. Nevertheless, to date the programme has had less of an impact in connection with these interventions. The evaluation notes that such training and investments require a significantly longer period of time to have an effect.
  • Also in the case of sustainable natural resource management, a component with a low implementation rate, impact will come about over the longer term. The target groups gave low priority to measures with barely perceptible results. A poverty reduction programme such as PRODEVER can have only a modest impact given the serious degradation of natural resources in the area.

The programme's gender approach was properly oriented towards creating conditions to reduce inequality between men and women in terms of participating in decision-making, access to resources, and the use and control of resources. Until the end of 2007, the gender approach, with no budget of its own, was successfully incorporated into the programme through the determined efforts of the programme coordination unit from the outset. Gender considerations were built into all PRODEVER activities, and gender training was provided to technical teams and implementers. In addition, a picture of the number of women participating in the programme was formed thanks to disaggregation by sex of most programme data, and a record of women's presence in most project documentation.

Indigenous peoples. PRODEVER has been implemented from the outset in a fully indigenous area with a majority presence of three indigenous groups: achíes, quechíes and pocomchíes. However, the intervention strategy addressed primarily their poverty rather than their ethnic condition. It was assumed that no special strategies would be needed to include the indigenous population as beneficiaries, nor was there a perceived need to thematize their rights.

Sustainability. The programme has placed priority on promoting grass-roots organizations in poor communities to enable them to move towards self-management, empowering their capacity to manage projects and generate resources. Under the local capacity-building component, 35 organizations (of 158 rated) were deemed consolidated in 2007. Among them are 27 producers' associations for commercial crops (coffee, cardamom, cacao, fruit), whose sustainability will depend on future demand for their products. Public and private institutions in the area of associations is still weak (municipalities have low resource levels and planning capacities, vested interests have a strong influence, etc.), and the sustainability of community structures requires continued support with links to emerging structures (development councils, municipalities, networks of organizations). PRODEVER faced constraints in providing adequate support: FONAPAZ project planning and implementation procedures, subject to state guidelines, led to discontinuity and at times poor timing of support, undermining the sustainability of their impact on production lines and food security. 

Innovation. In view of the baseline situation in Las Verapaces, promoting a participatory approach was innovative for the area. Moreover, new technologies and production alternatives were introduced (fruit, tourism), and more advanced linkages were established for product commercialization.

Performance by actors. IFAD has made a substantial investment in defining the programme and has monitored its implementation closely, visiting the programme area at least twice a year. In key areas such as developing and implementing an M&E system, IFAD made external consultants available from the beginning and closely monitored their performance. At the level of policy dialogue, IFAD has had a marked influence on the Ministry of Agriculture, Livestock and Food, FONAPAZ and other government agencies, including the Office of the President of the Republic. This work has borne fruit, first, in the form of capacity contributed to define models and approaches through projects such as PRODEVER, and then in setting strategic guidelines to promote rural development, as evidenced in the two IFAD COSOPs of 2003 y 2008. Guatemala's President, Oscar Berger, visited PRODEVER six times, demonstrating the Government's appreciation for IFAD's efforts in the area. In addition, the new Government declared rural development one of its main priorities in January 2008, creating the Rural Development Council and a coordinating agency represented by the National Rural Development Programme.

The Government has fulfilled its financial obligations. Following a prolonged preparation process, FONAPAZ competitively recruited a programme management and team with sound professional skills and kept it in place following the change in government in 2004. Despite FONAPAZ support for PRODEVER's work, in the absence of decisive governance, programme management decisions were accepted and the lack of a steering committee (provided for in the IFAD loan agreement) was tolerated.

Organization of the Petroleum Exporting Countries, as cofinancier, has not been involved in programme implementation, relying on the monitoring and supervision capacity of IFAD and the United Nations Office for Project Services (UNOPS) as cooperating institution.

UNOPS was contracted as a cooperating institution to monitor compliance with the loan agreement. In general, it performed satisfactorily. At the end of 2007, by IFAD decision, UNOPS was replaced in this function by the Andean Development Corporation (CAF).

Rating summary

Evaluation Criteria

Ratings

Core performance criteria

 

Relevance

5

Effectiveness

4

Efficiency

3

Project performancea

4

Rural poverty impact

4

Household income and assets

4

Human and social capital and empowerment

4

Food security and agricultural productivity

4

Natural resources and the environment

3

Institutions and policies

5

Other performance criteria

 

Sustainability

4

Innovation, replication and scaling up

4

Overall project achievementb

4

 

 

Performance of partners

 

IFAD

5

Government

4

UNOPS

5

a  Average of ratings for relevance, effectiveness and efficiency.
b The overall project achievement rating is calculated based on the ratings for project performance, rural poverty impact, sustainability and innovation.
Rating scale:  6 = Highly satisfactory; 5 = Satisfactory; 4 = Moderately satisfactory; 3 = Moderately unsatisfactory; 2 = Unsatisfactory; 1 = Highly unsatisfactory.

Conclusions

The following conclusions summarize the lessons that may be drawn from PRODEVER's design and implementation.
When PRODEVER was designed, the baseline situation presented a series of challenges to promoting development: a recent history of civil war with a significant impact on Las Verapaces and serious human rights violations, particularly against women; a population composed of different ethnic groups speaking different languages, with minimal relations among them; land distribution characterized by large estates in the hands of latifundistas or large landowners, with campesinos relegated to marginal land; power structures dominated by local caudillos; the lack of a central State presence; fledgling municipalities; communities isolated by inadequate infrastructure and services; informal markets and financial services; mass migration by young people; and a high incidence of poverty. Clearly, this situation entailed a series of risks in terms of design and implementation if the desired results were to be achieved, particularly in identifying which issues should be included in the programme.
Arriving at a design that enables implementers to respond to the complexity of such a situation means coming as close as possible to the actual circumstances in communities. PRODEVER's implementation has shown that in this situation, design by external missions has limitations, and that a presence is needed for a longer period to determine what is feasible to carry out. Conscious of the risks involved in designing PRODEVER, the designers requested the flexibility to adapt the design during programme implementation and IFAD agreed, adopting the flexible lending mechanism modality. This decision has been shown to be correct, since one component (rural financial services) and aspects of another (land titling) were not implemented, and new elements were included such as literacy, reproductive health, food security projects and vocational training.
IFAD required a clear programme focus and agreed to a design distinguishing among six different target groups. Following an initial study, the management defined poverty characteristics and levels in the communities to be served and selected the communities accordingly based on the nature of the projects implemented. However, no record was kept of the characteristics of the families and persons participating in the projects. As a result, given the stratification of results, it is not possible to accurately state to what extent each of the six target groups have benefited.
The design included two kinds of target populations: those with the means of production and potential for productive self-development, and those living in extreme poverty without such resources. PRODEVER has demonstrated that the most visible results were achieved in the segment of landowning families, who have plantations of perennial crops such as coffee, cardamom or fruit (banana, citrus), and those who were able to establish a permanent relationship with an exporter to sell vegetables. On the other hand, in order to reach target groups in the second category, the management designed projects not originally included, in particular in the food security sector. For this more numerous second segment, however, a significantly longer time is needed to cross the poverty line. Serving these different target populations calls for different approaches, methods and measures. These are not easily combined, and the programme has not differentiated clearly among them.
Additionally, the evaluation points up the fact that building local capacities requires an extended period of support. The programme results show that of the 158 organizations supported since 2003, 35 were deemed to have reached a level of development that merited rating them as "consolidated institutions". The majority of those remaining need continued support to achieve that status.
The programme objectives set initially were achieved in part, with notable progress made on organization, economic development, gender and road infrastructure. Substantial investments were made in improving conditions of access to markets for rural producers, such as roads and agro-industrial infrastructure, and much work was done on market access. An increasing economic dynamic was observed in Las Verapaces during programme implementation, to which PRODEVER made a substantial contribution in the form of training, technical assistance and investment in production and roads. Poverty was reduced by 5 per cent of the target population, coinciding with the national trend announced by the Government at the end of 2007.
The proposal to use the flexible lending mechanism was a sound one, as it allowed for the programme to be adapted on the basis of achievements and experience. However, this flexibility – indispensable for a programme such as PRODEVER – not only brought benefits in terms of adapting the programme to better achieve its objectives, but also entailed risks in terms of its evolution and the monitoring of implementation. In fact, the logical framework, which was to be used to measure progress, was applicable only in part and required significant changes to reflect the programme realities (see mid-term review recommendations). Clearly, such a situation calls for proper monitoring by those responsible for implementation. In the case of PRODEVER, IFAD made frequent visits to the field and provided constant support for the development of an M&E system.
Promoting rural development calls for interventions at various levels to cultivate related strategy and policy, linked to the actual realities in the field, by facilitating inter-project exchanges. PRODEVER is an example of best practice in this regard. IFAD supported the responsible agencies, such as Ministry of Agriculture, Livestock and Food and FONAPAZ, in defining various rural development programmes, and made important contributions to government policy through the COSOPs of 2003 and 2008. In addition, it facilitated an exchange of learning and experience between IFAD projects and other agencies operating in the country and elsewhere in Central America.
In view of the complexity of implementing a programme such as PRODEVER, it was very important to recruit a programme management and team of good professional calibre, to define and implement the proper entry and execution strategy to achieve the programme objective. In addition, the team needed to acquire the knowledge to design and implement effective projects that would increasingly be adapted to needs. FONAPAZ selected the appropriate management at the programme's outset, which applied sound judgment in setting up the team and selecting co-implementers. In addition, the fact that this team remained in charge of the programme from the outset until the end of 2007 was one of the main reasons for its good performance. Continuity was key to the results achieved. The change in the team at the beginning of 2008 interrupted the learning curve, and a new team had to begin again to establish relationships with the communities, earn credibility, and acquire the knowledge to provide continued support.
During the programme's implementation, significant changes occurred in the institutional framework, when legislation on decentralizing the State was passed, assigning a crucial role to municipalities as new agents of development. Taking advantage of this new environment in which public and private structures are emerging, PRODEVER has attempted to support the integration of its activities into municipal development plans, in some municipalities more successfully than in others. Members of organizations supported by PRODEVER currently participate in these structures, including community development councils (COCODE) and municipal development councils (COMUDE), which serve as an essential nexus to better anchor the programme's efforts within local structures. Through this process, opportunities for citizen participation have opened up and are gradually being employed.
PRODEVER operated in an environment that had become virtually bereft of public services. In this context, the programme sought out new ways of encouraging development in Las Verapaces. The participatory approach was innovative for the Las Verapaces area, as was the initial rating of communities by development level to facilitate selection on the basis of poverty and potential. At the same time, new technologies were introduced to improve the production and productivity of commercial crops. Opportunities were created for a sustainable increase in income by linking producers' associations with processing plants and with more advanced links in the market chain.

Recommendations

Design and preparation of future programmes

It is recommended that consideration be given to modifying the preparation of future programmes similar to PRODEVER by giving more of a voice to stakeholders active in the programme area.
It is recommended that, from the outset, different approaches and methods be identified to support different types of target groups: those who have the potential to lift themselves out of poverty through economic investments, and those living in extreme poverty without resources who need social assistance, at least initially. Focusing attention on both types of target groups risks lowering efficiency and effectiveness in both cases. Accordingly, it is recommended that greater conceptual clarity be applied in defining and combining modalities responsive to the two categories and their different levels of resources to help them.
It is recommended that IFAD's loan agreements include a due diligence requirement with respect to the executing capacity of the implementing agency, evaluating requirements in terms of governance and administrative guidelines to be met by future projects and their degree of compliance to ensure smooth and successful implementation.
Regarding rural financial services, it is recommended that the search be intensified for alternative forms of bottom-up savings and credit promotion to underwrite small financial markets adapted to the habits of potential community clients (savings and credit associations).

Continuation of PRODEVER

It is crucial that PRODEVER, FONAPAZ and those involved in defining the current government food security programme incorporate lessons learned from the implementation of PRODEVER to improve their operations in the country's 45 poorest municipalities, considered a priority by the current Government.
It is recommended that IFAD and CAF, together with FONAPAZ, perform an analysis of PRODEVER's current operating and financial capacity, including government contributions, up to the programme's conclusion. In addition, consideration should be given to a review of the continuing components and subcomponents and those which are to be closed, identifying the target groups on which the programme should focus to enhance the sustainability of supported structures. Also, the logical framework should be adapted and simplified. Finally, it is recommended that annual financial planning be established up to the programme's conclusion, including government contributions.
To ensure the sustainability of PRODEVER's successes and to promote public institutions at the community and regional levels, it is critical that closer links be established between organizations supported by the programme and permanent structures now emerging in the area: municipalities, development councils and others at the communal, municipal and departmental levels.
The results facilitated by PRODEVER in Las Verapaces would be orphaned without the continuous support of another programme to build upon accomplishments. It is recommended therefore that an investment be made in a programme to continue strengthening the capacities supported to date.

Policy dialogue

It is suggested that policy dialogue be maintained and reinforced by setting an agenda agreed upon by the authorities, IFAD and other agencies, with periodic meetings and work plans that include a review of the application of the Government's rural development policies.

 

LANGUAGES: English, Spanish

Upper Mandraré Basin Development Project – phase 2 (2009)

Madagascar  
December 2009

Completion evaluation - Executive summary

Introduction

Evaluation objectives and process. The completion evaluation of the Upper Mandraré Basin Development Project – Phase II (PHBM II) is intended in the first place to fulfil IFAD's obligation to report on the impact and results achieved by operations financed by its members and partners. In the second place, it should help project partners' learning process, with a view to the preparation and implementation of other rural development interventions in Madagascar. The evaluation focuses on five main aspects: (1) the performance of the project (relevance, effectiveness and efficiency); (2) the impact of the project on rural poverty; (3) the sustainability of project results; (4) innovations, their replication and scaling up; and (5) the performance of IFAD and its partners. Following a preparatory mission in May 2008, the main mission took place from 9 June to 4 July 2008. It based its work on three sources of information: a review of available documentation; interviews with project stakeholders and partners; and field observation in 15 villages within 6 of the 11 communes covered by the project. A wrap-up meeting with the various partners was organized at the Ministry of Agriculture, Livestock and Fisheries (MALF) on 4 July 2008, allowing discussion on the mission's aide-memoire.

Country context. Madagascar is an island country situated 400 km off the mainland of East Africa, with an area of 587.000 km² and a population of 18.1 million (2007), 73 per cent living in rural areas and nearly 44 per cent under the age of 15. Population growth is still high (2.8 per cent per year). The per capita gross national product is US$375 (2007) and the average economic growth rate is 5 per cent (2004–2007). According to the Human Development Index of the United Nations Development Programme, Madagascar was ranked 143rd out of 177 countries in 2007. The average life expectancy of Malagasies was 54.7 years in 2005. Agriculture provides employment for 78 per cent of the working population and generates 28 per cent of the gross domestic product (2007). The proportion of the population living below the poverty line was 66.3 per cent in 2007, as against 69.6 per cent in 2001. In 2005, poverty still affected 73.5 per cent of the rural – mostly agricultural – population, as against 77.2 per cent in 2001. The low level of domestic agricultural production has been identified as the main cause of malnutrition and poverty.

IFAD's strategy in Madagascar. Between 1979 and 2000, IFAD's activities focused on increasing food crop production and improving food self-sufficiency, especially in rice and meat. The IFAD Country Strategic Opportunities Paper (COSOP) prepared in 2000 focussed on improving the rural poor's access to technology on the basis of producers' demand, boosting farm capacities - particularly by improving irrigation infrastructures, organizing producers into users' associations, and promoting animal or mechanical traction. It also anticipated support and capacity-building for local institutions and the State's decentralized services. The most recent COSOP (2007–2012), approved in 2006, is based on three strategic thrusts: (1) improving risk management (especially risks linked to farm production and insecurity of land tenure) and minimizing the rural poor's vulnerability by increasing their access to resources and services; (2) improving the income of the rural poor by diversifying agricultural activities and promoting rural entrepreneurship; and (3) increasing the professional capacities of small producers and their organizations.

The Upper Mandraré Basin Development Project was formulated in the wake of the food crisis of 1991 that severely affected the south of the country after several years of drought. The first phase (PHBM I from 1996 to 2000) focused on four communes (3,233 km²) with 39,800 inhabitants in 1996. The amount of the IFAD loan was US$6.9 million. Implementation of the PHBM I helped to remedy the situation in the project area by improving access (102 km of roads and tracks), installing and rehabilitating 2,306 ha of irrigation schemes, improving health infrastructures (four basic health centres built), improving livestock health conditions (23 vaccination runs) and taking the first steps to organize local inhabitants (133 farmers' organizations established). The Interim Evaluation of the PHBM I by the Office of Evaluation in 2000 revealed that the project zone was still marked by severe inaccessibility and periodic localized food insecurity because of major climatic variability, an economic and social environment that was still not conducive to sustainable development, endemic rural insecurity connected with cattle thefts, and very poorly developed services and socio-economic infrastructures. The PHBM II (2001–2008) was formulated and appraised in the course of 2000, with an estimated cost of US$23 million. It covered nine rural communes (8 600 km²), increased to eleven in 2003 after two communes were split in two, and included the four communes already covered by the first phase. It targeted the whole population of the intervention zone, with a total of 96,000 people (17,400 households), especially women, young people and landless farmers.

Objectives and components. The general objective of the project was to contribute to a reduction in rural poverty, especially through: (1) a diversification and increase in the rural population's income; (2) an improvement in the food security of rural households; and (3) a contribution to the rehabilitation and sustainable management of natural resources. The PHBM II has five components: (a) Local capacity-building (7.7 per cent of the total cost); (b) Support for local initiatives (46.6 per cent); (c) Support to financial services (4.6 per cent); (d) Opening up of the project area (28.2 per cent); and (e) Coordination, management and monitoring-and-evaluation (12.9 per cent). The project's line ministry was MALF and the United Nations Office for Project Services (UNOPS) was IFAD's Cooperating Institution in charge of loan administration and project supervision for most of the project implementation period.

Project performance

With regard to the boosting of local capacities, the results obtained basically concern: (i) implementation of commune-level local participatory planning, which led to the formulation of 135 village community development plans, consolidated into 11 communal development plans; (ii) the training and installation of 11 communal development facilitators and 132 village-level facilitators; (iii) the recruitment and training of more than 1,300 local experts to provide social and economic services for communities in various sectors of activity; (iv) the establishment of a communal development committee (CDC) in each commune and the boosting of its capacities, in order to provide a space for discussion and a representative consultative body that can act as an interface between the commune and local inhabitants; (v) the promotion of nearly 1,500 rural organizations with a wide range of economic and social mandates; (vi) literacy training of 6,600 adults (49 per cent women); and (vii) the promotion of rural communications (radio station), nutrition and community health, as development tools supplementing other capacity-building activities.

The support for local initiatives expressed in the village community development plans and communal development plans has been consolidated, notably through the financing of nearly 1,130 productive sub-projects concerning agriculture, livestock and the environment; the boosting of agricultural and livestock infrastructures (2,250 ha of irrigation schemes installed or rehabilitated, more than 6,000 agricultural tools subsidized, 46 vaccination runs installed etc.) and the development of economic and social infrastructures (15 km of community tracks, 21 capital works and 36 water points installed, 9 schools and 5 health centres built). In addition, these local initiatives have been backed up with cross-cutting support activities (introduction and extension of technical packages on nearly 20 varieties of cassava, rice, maize and sorghum; support for marketing, supplies and agricultural services; and the promotion of environmental activities).

With regard to opening-up the project area the results have basically concerned the rehabilitation of 140 km of provincial roads (79 km of these in partnership with the Sectoral Transport Programme) and the rehabilitation of 64 km of roads between communes. Regarding support to financial services, the project established a savings and credit union and nine branches in the chief towns of the various communes.

Relevance. The project design had a number of strengths that contributed greatly to the achievement of the results described above. First, the project adopted a participatory approach to the planning and management of local development, which was harmonized and well coordinated between the communal and village levels and boosted by complementary capacity-building activities (literacy training; information, education and communication [IEC] concerning health, nutrition and environmental education; the organization and training of producers; and rural communications). Then, the project mobilized sufficient resources (human, financial and technical) to provide real support for rural inhabitants' local initiatives as expressed in the village community development plans and the communal development plans. Finally, a good overall consistency between the project's objectives and the Government and IFAD strategies was maintained throughout its execution, thanks to the various adjustments and refocusing of activities supported by the supervision and mid-term review missions.

The design of the PHBM II also contained weaknesses that would limit the scale of impact in certain domains and the sustainability of the project itself. Firstly, the social infrastructure component (health, education and drinking water) was too small, despite the fact that its vital importance was stressed by the Interim Evaluation in 2000. There was a lack of reflection and real effort during project preparation to establish a solid partnership that could have made up for this weakness. Secondly, project design failed to take the decentralized technical services into account as particularly valid strategic institutional partners in project implementation and as priority targets for capacity-building. Thirdly, it neglected the communal level as a valid centre point for the planning and management of local development, instead of focusing the participatory approach solely on the village level. With these various considerations, project relevance is thus deemed moderately satisfactory.

Effectiveness. In terms of results, the project has generally performed well inasmuch as it has already reached, one month after completion, an overall financial execution rate of 96.4 per cent (in local currency) and an overall physical execution rate of 122 per cent. The IFAD loan was, at that time, disbursed for 97.07 per cent.  In terms of the achievement of specific objectives, the project has shown varying degrees of effectiveness, depending on the objective in question. Thus:

  • It has been very effective with regard to the objective of boosting local capacities for the planning and management of economic and social development, as is seen especially in: the good level of ownership assumed by the CDCs with regard to the participatory planning process, consultation, the search for partnerships and the management of local development; the firm rooting of CDCs within communal structures; the boosting of the collective and individual capacities of local communities through the establishment and training of farmers' organizations and IEC concerning health, nutrition, literacy training for adults; and the real involvement of local experts trained in the supply of local services for communities. However, the evaluation highlights, on the down side, the sluggishness of the village development committees and also of the associations and committees responsible for the use and maintenance of certain socio-economic infrastructures (health centres, watering points, tracks).
  • It has also been effective with regard to the objective of support for local initiatives, as is seen, on the one hand, in the increase in irrigated areas and the greater intensification on irrigated land thanks to a more secure water supply, and, on the other, in the improvement in farmers' agricultural equipment and the extension of new agricultural and livestock techniques through the promotion of nearly 1,130 sub-projects. The main weakness lies in the low level of support with regard to social infrastructures and access roads because project design underestimated the importance of these components. Moreover, most of the infrastructures established are not used and maintained as well as they might be because of the inappropriate approaches and procedures adopted for their installation, the lack of specific training for the committees charged with their management and the continuing inaccessibility of many villages, hampering full use of the infrastructures.
  • The project had a fairly limited effectiveness with regard to the objective of developing sustainable agricultural systems. Some strong points were observed especially concerning the development of irrigated rice cultivation and market gardening (horticulture) on the banks of water courses, with considerable use of manure, the running of an integrated agro-forestry trial on five sites and attempts – with varying degrees of success – to diversify and boost rainfed agriculture and livestock rearing (cassava, goats, cattle health). However, many weak points were observed as well. First, the development of irrigated rice cultivation is precarious due to the only partial water control at the irrigation scheme level1 and the individual plot level, making the irrigated areas vulnerable during the off-season and in years with poor rainfall. The development of gardening is also limited due to the location of gardening plots, most often along the banks of small streams and rivers with intermittent flow and reduced potential in terms of irrigable land, and the predominance of manual irrigation with watering cans. Then, diversification of rainfed crops was limited (with a predominance of cassava), and extension and integration of soil protection and rehabilitation measures into the rainfed cropping systems were absent, although such techniques could minimize the risks associated with fluctuations in rainfall. Furthermore, the livestock rearing systems promoted by the project remain vulnerable and are poorly integrated with cropping systems. Finally, natural resource management activities were of an ad hoc and demonstrative nature in the absence of an overall vision regarding the development and sustainable management of natural resources.
  • Satisfactory effectiveness was noted with regard to the objective of optimizing production by opening up the project area, improving access to markets and reducing post-harvest losses. Although road and access track rehabilitation works did not succeeded in fully opening up the area, they have clearly led to an improvement in the circulation of goods and people in the project area. They have thus contributed to: better access to communal markets, certain production zones and major consumer centres outside the project area; a marked reduction in transport times; an appreciable increase in the number of agricultural produce collectors and transporters; an increase in the amount of produce on local markets; and revitalization of these markets. Moreover, the remarkable proliferation of paddy hullers (more than 30) has led to a considerable reduction in post-harvest losses. These various aspects have led to a major reduction in transaction costs and thus to greater value of agricultural produce. However, there are still some weaknesses, particularly with regard to the incomplete and still precarious opening up of the project area2 and continuing difficulties in the supply of inputs and the marketing of garden produce.
  • With regard to the objective of improving access to local financial services, a moderately satisfactory effectiveness was observed. The establishment of the FIVOY savings and credit union and its nine branches was late (2004), but since then it has made access to financial services much easier for local inhabitants, as the results of the network show: a penetration rate of 15 per cent (compared with an anticipated rate of 5 to 10 per cent); 5,213 beneficiaries reached (compared with the anticipated 4,500) of which 43 per cent were women; 3,455 union members of which 26 per cent women etc. Moreover, the most vulnerable groups (women and farmers without land or cattle) have been effectively reached by the network, thanks to a varied range of products, some of which require only the moral guarantee of a support group such as Credit with Education, specifically adapted to the development of women's income-generating activities, and the Associative Working Capital Credit intended to finance the activities of small groups of producers. However, certain weaknesses can be seen, especially with regard to: a poor capacity to mobilize savings (less than 30 per cent of total deposits allocated); the very slow progress toward financial autonomy (now expected for 2011 instead of 2008); and the very poor coverage of villages outside the main towns of the various communes.

In conclusion, given the positive elements mentioned above and considering the initial situation in the project zone, the overall effectiveness of the project is deemed satisfactory.

Efficiency. The economic internal rate of return of the project's productive investments calculated by the evaluation was at least 27 per cent and the timeframe for capital recovery was estimated at less than seven years, so that the investments made over the period 2001–2007 had already been recovered by 2008. This exceptional economic performance is essentially a result of the low unit costs of works on irrigation schemes. Moreover, the attainment of a 122 per cent rate of physical execution when only 96 per cent of the budget was used is a further indication of the project's efficiency. However, the efficiency of the rural financial services promoted by the project was modest, due to unfavourable conditions in the project zone and the staff's poor technical and management skills. To sum up, the overall efficiency of the PHBM II is judged satisfactory.

Rural poverty impact

The PHBM II has had a generally satisfactory impact on rural poverty. However, the scale of the impact varies across the different impact domains.

Household income and assets. The impact noted at the household level is very satisfactory. Monitoring by the Rural Observatories Network in the area showed that between 2002 and 2005 the average per capita cash income increased by 74 per cent. This increase affected all social categories, but particularly households without land or cattle (with a 178 per cent increase) and small farmers with less than 0.30 ha of irrigated land (with an increase of 160 per cent), who formed the project's special target group. This increase in income has had a major effect on the state of housing and led to an improvement in household and agricultural equipment.

Food security and agricultural productivity. The impact in this domain has also been very satisfactory, thanks particularly to the rise in rice and cassava production, with increases of 92 and 118 per cent respectively between 2001 and 2007, and also to the promotion of onion and garlic production (620 and 160 tonnes respectively) intended mainly for sale. This was possible thanks to the application of a value-chain approach focusing on three commodities (irrigated rice, garlic and onion) benefiting from a surge in demand from the rapidly developing city of Fort Dauphin and the global price increase of cereals. However, livestock yields saw very little improvement, and diversification and improvement in rainfed crops have been limited by periodic rainfall deficits. Household food security has improved significantly, seen in the very marked increase in the proportion of self-sufficient households, which rose from 36 per cent in 2001 to 78 per cent in 2008.

Human and social capital and empowerment. The project has also had a very satisfactory impact on the boosting of rural inhabitants' individual and collective capacities through literacy training, improved communications, the organization of grass-roots groups and associations, and training. The participatory approach has also allowed a new culture of consultation and self-confidence to be established at all levels, together with greater social and economic integration of traditionally marginalized categories (especially poor farmers and women). Furthermore, women's social and economic status has seen a marked improvement. Women are now much more involved in the new dynamics of social and economic development, forming more than 40 per cent of the members of grass-roots organizations, 67 per cent of sub-project recipients and 50 per cent of credit service beneficiaries. However, while social development is clearly visible in some spheres, it is still hampered particularly by the extent of the needs that have not yet been met with regard to drinking water, health and education facilities and services. Moreover, the situation of young people has not seen any marked improvement, for lack of specific targeting mechanisms.

Natural resources and the environment. The project has started to have some noticeable effects in this domain through the growth of awareness and a certain environmental discipline, both individually and collectively, thanks to the awareness-raising and educational activities undertaken and the demonstrations carried out. However, the ad hoc and demonstrative nature of these activities, combined with the fact that not enough time has yet passed, means that so far no widespread impact can be observed. Moreover, some negative effects and impact could be caused in the medium and long term especially because of the failure to incorporate soil protection and rehabilitation practices into the rainfed cropping systems practised on hillsides, and the lack of a vision and good practices for the rational, sustainable management of soil and water resources on irrigated areas.

Institutions and services. The project has enabled a radical transformation of the local institutional environment to take place by boosting the capacities of communal structures (CDCs, communal councils), revitalizing these structures and setting up an agricultural service centre, a local microfinance network and an inter-communal land tenure office. On the other hand, its contribution has been very limited in the sphere of consolidating and revitalizing decentralized technical services.

Sustainability and innovations

Sustainability. Insufficient time has yet passed to make a proper assessment of this aspect. However, some favourable indications exist, concerning especially the ready assumption of ownership of local development planning and management on the part of the CDCs and communal councils, the enthusiasm of the majority of the farmers' organizations established, the development of a new awareness that is more favourable to the protection and rational management of natural resources, and the emergence of new local services. On the other hand, some negative aspects constitute real threats to the sustainability of project gains. These consist particularly in: the incomplete and still precarious opening up of the zone; the lagging behind of social development due to the insufficiency of educational, health and drinking water infrastructures and services; the precariousness of the improvement in agricultural production due to climatic fluctuations and the absence of rational development and management practices for soil and water resources; the poor efficiency of microfinance services; the low capacity of decentralized technical services to take over the project because of the poor opportunities for capacity-building offered to them by the project; and the fragility of the newly created service institutions (agricultural service centre and inter-communal land tenure office) because of their novelty.

Innovations. The project has enabled the introduction of a number of innovations, particularly concerning the large-scale implementation of a participatory approach to planning and management of local development; the introduction of the value chain approach and its integration into the participatory approach; the incorporation, starting at the design stage, of an exit strategy, and its implementation throughout the project through a programme for supporting the transfer of assets and knowledge; the on-farm trial of a new concept of integrated agro-forestry; the introduction of new plant and animal production techniques and crop diversification. However, the project missed a number of promising opportunities for innovation, especially the introduction of strategies and practices that are better suited to the specific climatic conditions of the project area in order to manage and develop soil and water resources, and the promotion of local sub-project management by   communes or farmer associations.

Performance of partners

The Government and its various agencies have performed satisfactorily overall. The Ministry of Agriculture, Livestock and Fisheries has assumed effective ownership of the project and shown a good capacity for piloting and supervision. It has provided substantial support to the project through its central and regional services, and given it sufficient autonomy. The Ministry of Finance and the Budget has provided counterpart funds as foreseen. Other ministerial departments have also provided proper support through their sectoral services and programmes (infrastructures, health, water supply, the environment), despite the prior absence of any solid partnerships. Regional authorities and communes have been actively involved in project implementation, placing it at the centre of their concerns. However, decentralized technical services (at regional and district levels) have not been sufficiently involved by the project. The project management unit has performed particularly well, mobilizing a large number of partners not initially anticipated, taking many valid initiatives that advanced the relevance and effectiveness of the project, and ensuring good planning and coordination of activities. Although the project's monitoring and evaluation system came into operation very late, it has been exemplary. However, the project office has shown little responsiveness to the challenges of sustainable natural resource management (water and soil fertility in particular) and the adaptation of rainfed agriculture to climatic fluctuations. It has also experienced some difficulties in financial management of the project. Service providers have given a moderately satisfactory performance on the whole, with variations depending on the provider. Thus, the specialized technical operators and the generalist operator have generally performed well, supplying qualified staff and working in harmony with the project team. However, the performance of consulting firms and businesses is judged moderately unsatisfactory because of a number of technical errors and the lack of innovation in the case of the former, and insufficient equipment, training and managerial capacities in that of the latter. IFAD was given an overall moderately satisfactory performance. The initial project design had several shortcomings, with regard particularly to the insufficient consideration of a number of important recommendations made by the Interim Evaluation, the failure to establish a solid partnership and the exclusive focusing of the participatory approach on the village level. However, IFAD showed sufficient flexibility and responsiveness to be able to rectify some of these shortcomings at the time of the first mid-term review, even introducing some fresh adjustments to take account of changes in the strategies of the Government and IFAD itself. UNOPS performed satisfactorily by regularly carrying out supervision missions with the same teams, resulting in the growing effectiveness of its services. Moreover, its recommendations often went beyond simple verification of the status of technical and financial execution of the project, to touch on deeper technical, methodological and economic aspects, thus helping to improve the project's effectiveness.

Conclusions and recommendations

Overall assessment. The assessment of the PHBM II is on the whole satisfactory. The strong points noted are: a good efficiency and effectiveness in achieving the objectives of boosting local capacities, supporting local initiatives and optimizing agricultural production; a major impact on household income, agricultural production and food security; and a good performance of Government agencies, especially the project management unit. The weak points lie basically in: the lesser effectiveness in promoting sustainable production systems; the lower impact on natural resources and the environment; and the uncertain sustainability of certain achievements.
Ratings Attributed to the PHBM II and its Partners by the completion evaluation

 

Evaluation Criteria Ratings3
Performance criteria    
            Relevance 4  
            Effectiveness 5  
            Efficiency 5  
            Project performance 4.7  
     
Rural poverty impact 5  
            Household income and well-being 6  
            Food security and agricultural productivity 6  
            Human and social capital and empowerment 5  
            Natural resources and the environment 3  
            Institutions and policies 5  
     
Other performance criteria    
            Sustainability 3  
            Innovation, replication and scaling up 5  
     
Overall project achievement 5  
     
Performance of partners    
Government (Government agencies & PMU) 5  
IFAD 4  
UNOPS 5  
Service providers 4  

 Source: Completion evaluation of the PHBM II.

Recommendations. The completion evaluation of the PHBM II makes four recommendations. The first three concern the important strategic lessons that can be drawn from the experience of the project and should be taken into account in future rural development projects in Madagascar or elsewhere, while the fourth is intended to capitalize on the gains of the PHBM II.

Recommendation 1. To promote the balanced, sustainable development of Madagascar's rural zones by harmonizing and further integrating the interventions of integrated participatory local development projects with those of national sectoral programmes and regional development programmes on the basis of clear, firm partnership commitments.

Recommendation 2. To promote the diversification, integration and competitiveness of agricultural activities by promoting a diversified and integrated value chain approach to agricultural development, while placing women and young people at the centre of such development.

Recommendation 3. To promote the flexibility of project design and implementation so that implementation modalities can be modified and fine-tuned according to the changing context, thus allowing a good level of performance (relevance, effectiveness and efficiency) to be achieved and maintained, and a greater assumption of ownership of projects by those actually carrying them out.

Recommendation 4. To rapidly consolidate the dynamics of social and economic development already set in motion by the PHBM (I and II) in its intervention area in order to make the project gains permanent and capitalize on them.


1/ Fed by off-take diversion dams, offering no possibilities for water storage and control, and lacking adequate water control works or water distribution arrangements that could ensure a fair distribution of water among irrigators.

2/ The existence of a problematic un-rehabilitated 20-km section on provincial road 107, the destruction of an important ford at PK8 on provincial road 107 at the southern border of the zone, the existence of a 24-km un-rehabilitated section at the end of provincial road 117, which could have opened up the west coast zone, and the lack of upkeep of the roads and tracks already rehabilitated or constructed.

3/ Rating is carried out on a scale of 1 to 6 (6 = highly satisfactory; 5 = satisfactory; 4 = moderately satisfactory; 3 = moderately unsatisfactory; 2 = unsatisfactory; 1 = highly unsatisfactory)

 

LANGUAGES: English, French

Uplands Food Security Project (2009)

Korea, D.P.R.  
December 2009

Interim Evaluation 
Introduction

Evaluation objectives, methodology and process. In the spring of 2008, the Office of Evaluation (OE) of IFAD undertook an Interim Evaluation of the Uplands Food Security Project (UFSP) in the Democratic Peoples' Republic of Korea (DPRK). The main objectives of this evaluation were to (i) assess the performance and impact of the project; and (ii) generate a series of findings and recommendations for future projects and programmes financed by International Fund for Agricultural Development (IFAD) in the country. The evaluation adopted the latest methodology for project evaluations developed by OE, focusing on four areas: (i) performance of the project measured in terms of relevance, efficiency, and effectiveness; (ii) rural poverty reduction impact according to five impact domains; (iii) other performance criteria such as sustainability and innovation, replication and scaling up, and (iv) performance of the partners, including IFAD, the Government of DPRK, United Nations Office for Project Services (UNOPS) and other financial and implementation partners. The mission held talks in Pyongyang with Government partners such as the Ministry of Agriculture (MoA), including the Project Management Unit (PMU), the Ministry of Finance (MoF), the Ministry of Land and Environmental Protection (MoLEP), the Central Bank (CB), and the National Academy of Agricultural Science (NAAS). Development agencies and donors, e.g. The World Food Programme (WFP), FAO, UNICEF, the Swiss Agency for Development and Cooperation (SDC), and the European aid coordination office, were also contacted by the mission. In Ryanggang and North Hwangae Provinces, the concerned county officials of cooperative farm management coordination committees (CCFMCs) and of the CB branches accompanied the mission. The mission visited 13 Cooperative Farms (CFs), six in Ryanggang Province and seven in North Hwangae Province. The mission independently chose the CFs and the households for the conduction of interviews. This provided a good opportunity for the triangulation of otherwise available information and the discovery of additional aspects.

Country and sector background. DPRK is a country of 122,762 square km, with a population estimated at 23 million people. DPRK has not published official statistics in the past 30 years. The few available data on its economy, health, nutrition and agricultural resources are almost always estimates made by research institutions abroad. According to the most recent estimates (2004), the country had a GDP per capita of US$546, with an annual growth of 4.15 per cent. Agriculture was estimated to account for 18.4 per cent of GDP.

The sector went through collectivisation in the mid 1950s, which brought about mechanisation, the systematic use of chemical inputs and a national irrigation network. Cooperative farms, approximately 3,000 today, and state farms were created at this time. This doubled agricultural yields up to the mid 1980s. Agricultural productivity has since fallen, due to a variety of factors, such as overuse of chemical fertilisers and concurrent soil acidification, and widespread mono-cropping of staple crops, particularly on marginal soils. With the collapse of the Soviet Union in 1990, DPRK ceased to have privileged markets and access to cheap energy and other key inputs, which also affected the high input agriculture practised until then.In 1995-97, a series of droughts and flooding exacerbated the situation leading to widespread famine. The country had to rely on WFP food supplies for about 10 years until 2005 when it declared that no further emergency aid was needed but development assistance would be accepted. With less than 20 per cent of its land areas being apt for agricultural use, pressure on sloping land and forests continue to be substantial. Since 1990, DPRK lost 20 per cent of its forest area, and 10 per cent of the national food supply is estimated to originate from deforested sloping areas, while the official agricultural production apparatus exclusively relies on state and CFs.

Evolving agricultural policy context. With reduced support from neighbouring socialist states and scarce availability of external inputs since the early 1990's, the national strategy and overriding philosophy of self-reliance (Juche) has been revived with regard to agricultural policies. Greater emphasis is now placed on matching crops to soil characteristics, season and climatic conditions across the country. Focus has moved to improved seeds, intensifying production with double cropping, use of organic and bio-fertilisers, as well as bio-pesticides, that can be produced in the country. Mechanisation is still seen as an important priority as is the expansion of irrigated areas and reduced reliance on power for irrigation1 . State-owned, centralised and provincial input supply schemes, as well as state agricultural marketing organizations that feed into the Public Distribution System or export channels are all part of DPRK's agricultural policy. In July 2002, the Government of DPRK operated an economic policy shift on many levels that affected project implementation and impact. CFs were granted more autonomy in production planning, which facilitated the introduction of new crop rotations recommended by the project. However, the national currency (KPW) was devaluated about 70 times against the US$ and administered prices and wages were adjusted by a factor ranging from thirty to eighty. CF household cash reserves or bank accounts were not adjusted, nearly cancelling household debts and savings

Project background. DPRK is neither a member of the World Bank nor of the Asian Development Bank. Therefore, IFAD is at the moment the only International Financial Institution lending to the country. Since the beginning of its operations in 1995, which coincides with the onset of the large food crisis, IFAD has financed three projects in DPRK for a total loan amount of US$69 million. The Sericulture Development Project ran from 1996 to 2002 and the Crop and Livestock Development Project from 1997 to 2003. A Country Strategic Opportunities Paper (COSOP) was produced in 2000. According to this document, IFAD operations were to concentrate on reviving production in the disadvantaged uplands with focus on specific geographic areas, CFs and eventually households within CFs.

The project area included 46 cooperative farms in four counties. Two of these counties, Samsu and Pungso, are in the far north of the country, in Ryanggang Province; while the other two, Singye and Goksan Counties, are located in the south-east of the capital in North Hwangae Province. Of the 46 cooperatives, 37 cooperatives were involved in all components and nine were included only within the environmental preservation and credit components.

Project performance Design features.

The Uplands Food Security Project (UFSP) was approved by the Executive Board of IFAD in December 2000. It became effective in April 2001 and will close in June 2008 after two extensions, with the original closing set for December 2006. The goal was to implement balanced, sustainable and replicable cropping systems and environment management, which improve soil fertility and enable higher and more secure production to lead to improved living standards for 18,000 low-income households on 46 cooperative farms in upland areas, directly benefiting 61,000 persons. The project has seven components which correspond to the project's seven specific objectives (see table below). The total estimated cost at appraisal was US$41.77 million of which US$24.44 million from IFAD loans, US$4.44 million from the DPRK Government, US$7.18 million from co-financiers (WFP, FAO, United Nations Development Programme (UNDP) and Cooperazione e Sviluppo Italiana (CESVI)) and US$5.71 million from beneficiaries. UNOPS was assigned the function of Cooperating Institution (CI) for loan administration and project supervision.

Project components and specific objectives

Component       Specific objective Intervention strategy Per cent base cost
Sustainable crop production systems       Improved crop rotations, farming practices and soil fertility, generating increased yield, income and labour productivity. Assist CFs in introducing sound crop combinations and sustainable farming systems, providing models for demonstration to other CFs 55
Potato seed supply development         Greater availability to cooperative farms of high quality and disease free potato seed. Assist the Potato Research Institute and the participating counties' tissue culture centres;
Provide support to multiply the expanded seed volumes
2
Environment preservation       Improved micro-catchment planning, fuel wood plantations and erosion control measures preserving and enhancing the environment. Support tree planting, bunding, rudimentary terracing and the construction of storm drains 31
Household and cooperative credit       Credit services encouraging and enabling livestock and other enterprises by cooperatives and their farmer members. Provide additional institutional credit to finance investment in income-generating production activities, mainly livestock 4
        Capacity of cooperative communities to select and undertake productive projects. Establish a fund to stimulate the process of participatory planning and investment at the farm level
3
Farm output processing       Improved processing capacity adding value to crop production at cooperative farms. Help the cooperatives to investigate and develop opportunities for processing farm products 1
Project implementation support       Technical and managerial capacity of national, provincial and county agencies and cooperatives to plan and implement projects. Provide support to PMU, Provincial Rural Economy Committee (PREC)sa and CCFMCs in carrying out their roles. 4

 

a Provincial Rural Economy Committees
Source: Appraisal report

Implementation results Sustainable crop production systems component (61.1 per cent of project cost at completion). The core element of the component was the introduction of sound crop rotations on 18,000 ha in the 37 CFs with full project support, supplemented by the supply of annual crop inputs, such as fertilizer and pesticides, and the provision of mechanisation packages. After an initial delay caused by a misunderstanding of the project approach, improved crop rotations are now mainstream in the supported CFs.

Farm input supply was halted in 2005, on the basis of an Mid-term Review (MTR) recommendation, as the fertiliser procured by the project was not systematically applied to the soil classes foreseen at appraisal and the project inputs mostly replaced inputs formerly obtained by CFs from other sources. Farm machinery was successfully supplied to the CFs, however without taking into account the extreme difficulties of the CFs to procure spare parts and tyres, for lack of adequate maintenance capacity and of access to hard currency. Farm management support was provided through training, coaching and study tours for CCFMCs and CFs.

Potato seed supply development component (2.6 per cent). This component addressed the lack of high quality and virus-free potato seed in the project area. The project provided inputs and equipment to the Potato Research Institute at Daehongdan in Ryaggang Province and supported renovation and equipment for four county level potato tissue culture factories, where potato foundation seed is multiplied in-vitro in greenhouse multiplication units. The project also supported field level propagation of potato seeds in 37 CFs. Specialists from NAAS were actively involved in implementation.

Environment preservation component (14.3 per cent). MoLEP was responsible for the coordination of this component. Five thousand five hundred ha of fuel wood plantations for 46 CFs have been planted and/or replanted against a target of 5,400 ha. This was possible, despite the very limited funding from WFP in the form of Food for Work 2. MoLEP also reports having undertaken bunding, terracing and construction of storm drains for the protection of 1,200 ha of sloping land, i.e. 50 per cent of the target at appraisal.

Household and cooperative credit component (15.6 per cent). CB was given the overall responsibility of implementing the credit scheme whereby 80 per cent of the earmarked funds were for households, and 20 per cent for CFs, which raised small livestock that was outsourced for further rearing and/or fattening to the households in a given CF. In total, more than 45,000 households benefited through repeated loans, with a balance of 17,000 outstanding loans by the end of 2007, i.e. according to the appraisal target. Based on this success, the project funds allocated to this component were doubled during project life, and the portion earmarked for CF credit was raised to 50 per cent. However, the revolving fund for household and CF loans was not adjusted when administered prices and wages were dramatically increased in 2002, thus losing most of its value. A reallocation of US$2.23 million of unspent loan funds to the household credit fund was agreed one month before project completion.

Community facilities and services component (3.6 per cent). This component was implemented from 2006 onwards only, on the basis of a training of PMU and CF staff in Participatory Rural Appraisal (PRA). The delay was caused by the absence of UNDP funding for hiring international technical assistance (TA) to train CF managers and members in participatory planning techniques. It was only when IFAD ultimately agreed to provide a grant that the training could take place and the component could successfully be implemented. CF management and members were enabled to prioritise community facilities and services and to submit respective projects to the PMU. The project funded building materials for 163 community facilities, ranging from cultural centres, kindergartens, clinics, bridges, and threshing sites.

Farm output processing component (1.2 per cent). The Appraisal Report required the project to carry out feasibility studies covering all processing opportunities to add value to farm outputs.

These feasibility studies were conducted with a long delay in 2006, again because UNDP funding originally committed for those studies was not forthcoming. The studies were ultimately carried out with an IFAD grant. Starting in 2006, rice mills, oil presses, feed crushers, noodle makers and potato starch plants were procured, totalling 102 sets of machinery. 

Project implementation support component (1.6 per cent). The PMU managed the procurement of machinery, equipment and farming inputs and assured their distribution through the established channels of MoA. In addition, the PMU was responsible for project coordination, training and monitoring and evaluation (M&E) activities, and for convening the monthly Steering Committee meetings. Nine short-term TA missions were conducted over the life of the project, which was significantly less than planned due to FAO, UNDP and CESVI financing not being fully tapped for these inputs, and the reluctance of the Government to finance international TA out of loan proceeds. Trainings on M&E, procurement, environmental planning, whole farm planning and project reporting were financed by UNDP. WFP financed TA on Food for Work project assistance planning and IFAD financed TA on PRA and agro-processing. Only the very last TA mission, on Financial Management, was financed from the IFAD loan. In terms of M&E, the PMU was successful in conducting a household survey with repeats on the same household sample at regular intervals, and in establishing a CF performance database of the participating CFs. However, the PMU had considerable difficulty to obtain timely and comprehensive project information from the partner institutions, especially the CB. This made the task of the evaluation mission particularly challenging. As per 31 May 2008, US$35.45 million or 84.9 per cent of the original financial outlay had been used. At loan closing (31 December 2008) the IFAD loan was 100 per cent disbursed. 

Performance assessment

Relevance. It is fair to say that the project objectives were consistent with the agricultural and rural development policy of DPRK and with the IFAD COSOP. Project design included learning from previous IFAD projects in DPRK. The appraisal report integrated relevant inputs from the IFAD internal quality enhancement and quality assurance processes. However, because of the design missions' limited access to information, project design documents remain superficial in terms of analysis of causes of rural poverty, constrained agricultural productivity, environmental degradation on sloping lands and so on.

As a result, the technical orientations of the projects' main component, sustainable crop production systems, were partly inappropriate and did not address three main issues, i.e. the lack of farm machinery maintenance capacity, the need for alternative agronomic solutions to substitute imported inputs and widespread soil acidity. Other project component were considered relevant although the environment preservation component focussed mainly on reforestation of sloping lands and did not promote sustainable use of sloping lands, although the food security of an important part of the rural population living outside CFs depends on those lands. Overall, project partnerships were weakly developed during project design, both with national and international institutions. The involvement of co-financiers was well intentioned, but inadequately prepared and not secured by formal agreements between the borrower and those co-financiers as foreseen by the Loan Agreement. This lead to the nearly complete failure of honouring expected project participation as outlined in project design documents. Finally, project design did not take into account the serious communication issues between the PMU, IFAD and other international development partners, which still prevail today. 

Effectiveness. After a four-year delay caused by misinterpreted appraisal recommendations and an inappropriate top-down approach, improved crop rotations were successfully introduced in the supported CFs (on 31,000 ha against a target of 18,000 ha) and had a positive effect on soil fertility. The insertion of fodder and green manure crops into the rotation is taking place on an experimental basis in North Hwangae Province only, and not systematically in all rotations as expected at appraisal, probably due to the national priority given to cereals and potato production for human consumption. The provision of chemical fertilizer did not have the desired effect on soil fertility because of prevailing soil acidity, which was probably exacerbated by an unbalanced fertilizer mix.  The other project components were effective: 12 million disease-free potato mini tubers were produced during project life and provided to project farm cooperatives. The required number of high quality potato tubers has apparently achieved area coverage, as verified by the mission in Pungso County.

Environment preservation activities have been delayed by structural coordination problems between the project, MoA, MoLEP and WFP, and the practical absence of WFP funding. However, in the end, the target of 5,400 ha of wood lots has been overshot, with MoLEP and CF contributions for the major part. Survival rates were reported as quite high in North Hwangae (some 80-90 per cent), but can be significantly lower in Ryanggang, particularly in poor rainfall years (down to 50 per cent). Anti-erosion measures attained 50 per cent of the planned 2,400 ha.

The credit component was implemented effectively by CB and CFs, with zero default rates and for the anticipated number of beneficiaries by the end of 2007. Women were the main borrowers for household credit, receiving about 90 per cent of the loans. The participatory planning approach for community investments introduced by the project was successfully applied and all 37 CFs concerned have been able to select, plan and implement community facilities and services in a participatory manner, although with a substantial delay. Farm output processing facilities installed since late 2006 appear to be in regular use.

Efficiency. Evaluating project efficiency in economic or financial terms in DPRK is particularly challenging because of the scarcity of statistics and comparative data and the administered nature of costs and prices. Therefore, no cost benefit analysis was attempted by the mission, but efficiency was addressed in other terms. The procurement of farm inputs and machinery under the sustainable crop production systems component, the credit component and the potato seed supply component were implemented according to schedule. However, implementation was slow during the first half of the project for all other (sub-)components. The main reasons for delays incurred were: (i) the lack of clarity or inappropriateness of certain design features, in particular concerning the largest project component (sustainable crop production systems); (ii) the meagre contribution and early withdrawal of international project partners (WFP, FAO and UNDP) due to the lack of formal of agreements at project start-up and, subsequently, the poor communication and coordination between the PMU and project partners; and (iii) the late decisive involvement of national partners such as NAAS and provincial agriculture universities. Consequently, the MTR recommended a one-year extension of the project's closing date. The project suddenly shifted gears, by allowing CFs to prepare their own crop rotation plans and field trials with guidance from NAAS scientists and the CCFMCs, and by mobilizing IFAD grant funds for further TA to the project. As a result, the project succeeded in making up for most delays by mid-2007. A second extension to the project was agreed following the supervision mission of 2007, to allow the project to: (i) internationally procure farming supplies for US$1.5 million approximately, to be sold to CFs, the revenue of which would be allocated to the household credit revolving fund with CB; (ii) benefit from technical assistance in financial management; and (iii) bring its financial records up to standard. Procurement prices of farm inputs and machinery, purchased mostly from China, were lower than international standards.

However, in connection with these two major cost items of the project, the mission perceives a low end-user efficiency, due to the limited plant nutrient availability in acid soils and the down-times of agricultural machinery for lack of spare parts and poor machinery maintenance capacity. This was to the detriment of the participating CFs, who had to pay for all items procured either directly or via standard loans. On the other hand, they all obtained farm inputs and machinery at the administered prices substantially below economic prices. An estimated US$1.2 million of project financing for the credit component was lost following the administered price adjustment of 2002 without adjustment of the revolving fund. Moreover, US$2.23 million of unspent loan proceeds were reallocated to the household credit component following the Governments request at the very end of the project without proper measures to ensure that the full value would reach the beneficiaries. The value of the amount reaching the beneficiaries in local currency is far below the opportunity cost of the amount in hard currency transferred by IFAD to DPRK.

Performance of partners

While the mission commends IFAD, as the only International Financial Institution (IFI) in DPRK, for it's support to the rural poor in this particularly challenging institutional context, IFAD performance was rated moderately unsatisfactory because of: (i) the failure to foresee and address communication difficulties; (ii) the inability to finalize co-financing arrangements with other development partners; (iii) the inadequate provision of continuity in country programme management and of implementation support; and (iv) the inappropriate reallocation of a large amount of loan proceeds at the very end of the project. The performance of UNOPS was also rated moderately unsatisfactory. The quality of supervision reports was acceptable overall, considering the difficulties met by supervision teams to access information and project sites. However, until the Mid-term Review, recommendations were sometimes inconsistent on key issues such as crop rotations, and often lacked clarity. The main causes for this were the lack of continuity of supervision team members and the insufficient time for proper discussion of findings and recommendations with project partners at the end of each supervision mission. UNOPS also overlooked several cases of procurement mismanagement by the project. As for Government performance, the policy shift in 2002 allowed for more autonomy in production planning for the CFs, which facilitated the implementation of component 1 in particular. Strong policy support helped the project to achieve its reforestation targets under the environment preservation component with full commitment from MoLEP and CFs alike, without the large volumes of Food for Work originally to be provided by WFP. However, the devaluation of the KPW coupled to the adjustment of prices in 2002, strongly reduced the real value of the credit revolving funds. Also, efforts by MoA, as the responsible line ministry of the UFSP, to support the establishment of partnerships between the project and national and international institutions in DPRK were insufficient. All through the project life, information was managed in a very restricted manner and UNOPS and IFAD missions were not readily granted access to the field. Limited freedom of movement and scarce access to background data for project designs missions have handicapped the project design process, and thus affected project relevance. The project beneficiaries deserve a highly satisfactory performance rating. The success of the household credit component is the result of much extra work above and beyond the regular duties of beneficiaries in their CFs. 

Rural poverty impact and other performance criteria

Rural poverty reduction impact. Household income and assets increased significantly over project life for about 20,000 households in the participating 46 CFs, with a concomitant increase in purchasing power, all directly induced by the household credit component. Savings also increased over time, but mostly under the form of household appliances because of the limited opportunities for households to reinvest income in productive assets, as current Government policy restricts individual economic activities in CFs, and the risks related to monetary saving. By strengthening economic activities at their own risk and responsibility, the households gained social capital and empowerment, especially women who were the clients of 90 per cent of the household loans extended. Food security was enhanced in two ways, first by increased agricultural productivity, which allowed higher farm dividends to be paid to the households, and second by meat and milk produced at household level, thanks to the credit component. Wood lots planted under the environmental preservation component now correspond, in average, to 0.3 ha per household, which is deemed sufficient for annual fuel wood requirements. However, these achievements are marginal the face of past and continuing encroachment of forest and cultivation of sloping land, which remain a matter of concern for the natural resources and the environment of DPRK. In terms of institutions and policies, some relaxation of the rigidly planned agricultural apparatus is now visible, to which the project has not directly contributed but rather was in position to take advantage of. Overall, rural poverty reduction impacts are rated as satisfactory.

Sustainability. Factors that enhance sustainability are the following: households, particularly women, have been enabled to assume risk and responsibilities. This is the essence of social capital and empowerment, which stands a fair change of being sustainable.

The development of a livestock breeding and fattening system, in tandem between CFs and households, has generated assets, income, savings and food security. The widespread adoption of improved crop rotations is technically sound and very likely to be continued after the project. The concomitant shift from rigid top-down central planning to a more participatory method, albeit still within relatively narrow boundaries, bears the potential of unleashing additional human talent at various levels. In addition, the mission notes the likelihood of (i) sustainability of the potato seeds supply and (ii) continuity of the environmental preservation efforts. On the other hand, the UFSP is marred by a number of factors that constrain sustainability. The dependency from imported machinery, spare parts, inputs and energy is pervasive for the agricultural sector as a whole, and the USFP granted only a temporary and marginal relief. Moreover, the visibly continuing process of cultivating sloped land originally under forest, outside the official array of CFs, is an important environmental sustainability concern. Despite the achievements, the resource base of the CFs and households remains fragile. A consecutive series of bad harvests or new administered price adjustments could compromise the respectable impacts achieved by UFSP.

Innovation, replication and scaling up. The project promoted the introduction of a few important technical innovations (crop rotations, potato seed multiplication scheme) which appear to have been replicated outside project CFs. Field trials to further improve agricultural practices, although not directly promoted by the project, are underway. A downside with regard to further scaling up those innovations, is that research and development findings are not of the public domain in DPRK. An important innovation was present in the credit component, which promoted household level animal breeding and fattening activities, taking advantage of individual zeal and initiative, in complement and not in competition with collective farming activities. Also, the efficient manner in which the credit component was handled (most transaction costs were outsourced to the CFs) was well adapted to the cooperative system in DPRK and may be considered innovative. The successful credit component is, however, apparently not yet being replicated outside the project CFs. Current Government policy also restricts opportunities for CF households to develop individual economic activities, which constrains the further increase of benefits from household credit.

Conclusions

In general terms, the UFSP has contributed in attaining IFAD's strategic objectives: (i) Poverty alleviation by targeting rural poor in marginal areas with food security constraints; and (ii) Empowering beneficiaries through interventions such as household credit, natural resources management, community facilities and services and farm output processing. The project was relevant in the sense that it addressed food security and rural poverty constraints at the time of appraisal and chose marginal upland areas in a farming systems approach, very much in line with the COSOP. On the other hand, the design of the project's main component, contained some important flaws which were mainly due to a lack of involvement of project stakeholders and other local expertise at the design stage. Project design also underestimated the serious communication problems and failed to secure the participation of co-financiers. Despite the flaws mentioned above, the project was implemented with satisfactory effectiveness, although with important delays in the first project half. Efficiency was also affected due to the Korea Won (KPW) devaluation in 2002 which lead to the loss of about one third of IFAD funds invested in the credit component, and limited end-user efficiencies of input and machinery imports. Overall poverty reduction impact was satisfactory, notably in terms of household income and assets, social capital and empowerment and agricultural productivity and food security. The mission concludes that these achievements were possible to a large extent due to the CF and household credit component, which unleashed enthusiasm and talent and eventually generated the above mentioned impacts. Small livestock reared and fattened by households, in conjunction with an organized supply of young stock and close-by marketing channels, appears to be a winning formula for making important leaps in rural poverty reduction.

Tapping the talents of household as units of economic activity has particularly paid off: with only 10 per cent of actual project cost, the returns were broad-based, significant and harbouring a high potential of sustainability. However, a number of factors limit sustainability of project impacts, such as the continuing dependency on imports of agricultural production in CFs, the increasing cultivation of sloped land originally under forest, outside the official array of CFs, and the enduring fragility of the resource base of the CFs and households. The table on the next page summarizes the evaluation ratings of the UFSP, with an overall rating of project achievement of 4 (moderately satisfactory).

Evaluation Ratings of the UFSP

Evaluation Criteria Ratings
Project performance
Relevance 4
Effectiveness 5
Efficiency 3
Overall project performance 4
Rural poverty impact  
Household income and assets 5
Human and social capital and empowerment 6
Food security and agricultural productivity 5
Natural resources and the environment 4
Institutions and policies 4
Overall rural poverty impact 5
Other performance criteria  
Sustainability 4
Innovation, replication and scaling up 4
Performance of partners  
IFAD 3
UNOPS 3
Government 4
Beneficiaries 6
Overall project achievement 4

 

Recommendations

As the UFSP was an interim evaluation, the recommendations aim at setting a preliminary stage for the design of a subsequent IFAD operation in DPRK. Agreement on those recommendations and principles should be formalized in the forthcoming results-based COSOP – a necessary step before initiating the design of a follow-up IFAD investment loan. With this in mind, the evaluation makes the following recommendations:

Recommendation 1: Project design To ensure that future IFAD intervention in DPRK respond to the needs of the rural poor and propose sound and sustainable technical and institutional solutions to rural development constraints, the design process for future IFAD interventions in DPRK would require: (a) Ample participation by the envisaged target population, and its existing forms of organization; (b) Strong collaboration with national and international rural development partners; (c) The Government to grant full access to relevant information required for a sound project design; and (d) IFAD to mobilize its own resources to enhance its knowledge and understanding of the country and the needs of the rural poor for instance in the framework of the preparation of the new COSOP.

Recommendation 2: Partnerships These stand out as the key to development cooperation with DPRK, and therefore: (a) IFAD should give particular attention to enhancing its partnerships and building new collaborations with national and international institutions concerned with agricultural and rural development in DPRK; (b) The Government should actively encourage partnerships among national and international institutions and take up a coordinating role; (c) The Government should also promote communication and information sharing between the PMU and project partners, all through the project cycle; and (d) Project partnerships, including co-financing arrangements, should be carefully chosen and formally established with a clear distribution of responsibilities among partners, as early as possible in the project design process. 

Recommendation 3: Sustainability Environmental, technical and economical sustainability of rural development efforts and achievements should be given greater attention. In particular: (a) Environmental components in IFAD projects should focus not only on reforestation and protection, but also on sustainable and profitable use of sloping land, by the important part of the rural poor today that live outside the CFs; (b) The Government should consider the challenges of working the land and maintaining soil fertility in a context of very limited access to imports as an opportunity for developing alternative social production arrangements, such as centring the responsibility for agricultural production on autonomous but well supported sub-work teams within cooperative farm structures that would assume the function of service and credit providers; and (c) Innovative technical options to increase and maintain soil fertility on CF lands should be further explored by field trials and, if found adequate, divulged to CFs for generalisation. This recommendation could be initiated with the support of an IFAD grant complemented by technical assistance from national and international rural development partners.   

Recommendation 4: Household credit Considering its important impact on income, food security and empowerment of rural households, the household credit scheme could be scaled up to all CFs in DPRK, possibly with the support of a new IFAD intervention.

However, it would be necessary that: (a) The reporting system of CB towards the PMU and project partners be improved, and the bookkeeping system at farm level be standardized; (b) Additional degrees of entrepreneurial freedom for potential borrowers be explored and agreed upon; (c) The concomitant lending for small livestock to CFs, either through work teams or sub-work teams, remain an option, which may bear a significant potential for synergy with household credit.


1/ Agricultural Policy Document provided in translation by PMU on 1 May 2008.

2/ WFP only provided 8.7 per cent of the US$6 million promised at design, apparently because the PMU did not file any additional requests following correct procedure and because of serious communication constraints between WFP and the PMU.  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

LANGUAGES: English

Western Mindanao Community Initiatives Project

Philippines  
October 2009

Poverty in the Philippines is predominantly rural and Western Mindanao is one of the poorest and least developed regions in the Philippines. The agriculture sector accounts for over a third of total employment but production is not keeping pace with population growth. Low labour productivity characterizes the sector and a large part operates at subsistence level and is vulnerable to year-to-year weather changes.

About six out of ten people in rural areas depend on agriculture for their livelihood. Fishing is an important sub-sector, mostly at subsistence level. The current Medium-Term Philippine Development Plan (MTPDP) identifies Mindanao as an area of strong agricultural potential.

The project area is in a conflict zone. The conflict can be traced to centuries of discrimination, as perceived by the Lumad (indigenous) and Muslim people, that has marginalized them in terms of social and economic development. While a peace agreement was signed between the Philippine government and the Moro National Liberation Front (MNLF) in 1996, episodes of conflict in 2000 and 2003 have been costly, with the displacement of nearly one and a half million people, including those in project areas.  Despite the armed conflict, tri-communities co-exist in Mindanao (Lumads, Muslim ethnic groups, and migrant settlers from Visayas and Luzon).

The project has four components: (i) Community and Institutional Development, (ii) Natural Resource Management, (iii) Small Enterprise Development and credit, and (iv) Project Implementation. Total project costs were estimated at US$18.15 million of which the International Fund for Agricultural Development (IFAD) was to finance US$15.54 million (85.6 per cent), including a grant of US$ 0.75 million.

The goal of WMCIP was increased subsistence, higher incomes, better standards of living and greater resilience of livelihood of up to 16,000 farm and fishing households. Twenty one municipalities with 80 barangays ("village" level administrative unit) were pre-identified for inclusion at design stage (later increased to 81). The project was appraised during 1996-97 and approved in 1998, with an implementation period of six years; it became effective in 1999.  The original loan closing date was extended from Dec 2005 till Dec 2006, and now Dec 2007, with project completion 30 June 2007.

This interim evaluation followed the Office of Evaluation's (OE) methodology for project evaluation in assessing performance and impact. Its objective was to develop recommendations for enhancing the design and implementation of new and ongoing IFAD-funded projects, and to facilitate IFAD management's decision on whether or not a follow-up phase of WMCIP should be financed by the Fund. The Evaluation Team visited three of the four provinces covered by the project, Zamboanga del Norte, Zamboanga del Sur, and Zamboanga Sibugay.

Visits to Basilan and Tawi Tawi were not possible due to security concerns.  At the time of the field work, the self assessment by the government, project completion report, and United Nations Office for Project Services (UNOPS) final supervision mission report were not available, hampering the evaluation.

Performance

Design. The project components were designed to emphasize participatory planning and implementation and covered: community development and institutional capacity building, natural resource management and enterprise development.

It covered three supposedly contiguous sites where the most disadvantaged populations could be targeted, comprising upland areas, agrarian reform lowland areas, coastal areas, and indigenous peoples. Barangay project sites, however, were not contiguous, their wide spread locations making project implementation and supervision difficult. In addition, interconnectivity of upland-lowland-coastal ecosystems was not captured. The design, however, did focus on key environmental concerns that impacted on people's livelihoods: (i) over-exploited land resources, farmed in a haphazard manner, resulting in degradation and loss of soil fertility, affecting production and incomes; and (ii) depleted fish stocks from over-fishing and use of destructive methods and practices, resulting in damage to marine and water resources.

A pilot scheme to address concerns of vulnerable households was introduced in 2003, and expanded in 2005. The credit sub-component of the project was not implemented for three years, the design not being suitable to the needs of the project beneficiaries given the stringent lending policies of the Land Bank of the Philippines'(LBP - the implementing agency), and reluctance of credit conduits to participate in the credit program. Changes recommended by studies commissioned by the government to address the problem were not implemented.

Overall performance. By June 2007, the project achieved or exceeded practically all quantitative targets, apart from some infrastructure provision. Initial project start-up was delayed pending resolution of project management issues (para.18 below), while performance up to mid 2004, when the project was originally due to be completed, was slow. The project has been extended by three years. Over 9,300 poor farmer households and almost 2,400 poor fishermen households have directly benefited, including nearly 3,400 vulnerable households. Nearly 9,000 households are involved in the process of enterprise development. The draft Project Completion Report (PCR) states that households' income increased by 50 per cent to 75 per cent over six years. However, based on a survey conducted by the project in 2007, average annual income of beneficiaries has increased by about 38 per cent since 2005. This was largely due to higher farm incomes, attributable to the beneficiaries' adoption of new agricultural technologies under the project.

Community and institutional development. Involved the mobilization and participation of the community in the identification of development needs and in the prioritization of interventions responsive to the needs of the community. Targets for assisting community organizations were exceeded, and several innovative practices introduced are now considered as good practice. Local Government Units' (LGU) capacity was strengthened and partnerships developed with line agencies. The objective of improved community capacity to plan programs and access funds for the communities' priority projects was achieved.

Natural resource management. In land resource management, over 9,000 farmers were trained in appropriate farming technologies; nearly 8,000 adopted the technologies, conserving approximately 2,405 hectares. Project activities focused on conservation or regeneration of natural resources, new farming technologies reducing production costs and integrating short-term production (crop and animal) and long gestation crops in farming systems. They addressed nutrition, food security, environment protection, and improved productivity. Technologies introduced have been socially and culturally acceptable; training was provided to improve para-technicians' capabilities in transferring technology to other farmers in the community. If more beneficiary barangays had been contiguous, then benefits from the new technologies may have spread more widely.

In marine and water resource management, over 2,300 fishermen were trained in appropriate technologies, such as coastal resource management with built-in environmental protection features. Mangroves were rehabilitated, and artificial coral reefs installed; 289 hectares of municipal waters have been delineated and declared as Marine Sanctuary. The Marine Protected Areas, artificial reefs, and mangrove rehabilitation had positive effects on the marine environment and fish populations.

Indigenous people. The project in partnership with the National Commission on Indigenous Peoples (NCIP) helped security of land tenure, facilitating the award of Certificates of Ancestral Domain Claims (CADCs), and conversion to title (CADT).

Rural infrastructure. Slippage and time over-runs of infrastructure sub-projects occurred due to delay in fund releases, and to LGUs' absorptive capacity. Many sub-projects have not been completed, with over 40 not started; 12 sub-projects will be cancelled. Maintenance is a concern for some of the roads, which are beyond barangay capabilities. Sub-projects were chosen through community consultation meetings as part of preparation of the barangay development plans; however they were not then specifically reviewed to determine their feasibility and environmental viability.

Small enterprise development.  Overall numerical targets were achieved, but there is no evidence that beneficiaries are consistently engaged in enterprises and making reasonable returns, nor that effective government and private research and advisory services for on- and off-farm enterprises were successfully established. The LGUs and Non Governmental Organisation (NGO)s lacked capacity in business advisory services, and interventions did not produce desired results; only a few enterprises have potential. Few beneficiaries availed of credit, with low repeat credit availment. The savings and credit approach, however, was a positive project achievement.  Efforts to redesign the credit component did not materialize. Credit operations moved forwards towards the end of the project, but the project no longer had resources to provide advisory support.

Gender equality and mainstreaming. The proactive approach to include women in barangay and livelihood activities was very evident, with women's associations being formed in all project sites. However, instead of balancing and harmonizing roles of men and women, the project approach resulted in a compartmentalized perspective of gender and development.

Conflict and peace building. Natural Resources Management (NRM) partner agencies, NGOs-People's Organizations (POs), civil society and the armed forces converged in efforts to achieve the project objectives. The armed forces cooperated with civil society in the protection of the Buluan Fish Sanctuary. Support was extended by "leftist" elements in Zamboanga del Norte. In Zamboanga del Sur and Zamboanga Sibugay, Moro Islamic Liberation Front-Moro National Liberation Front (MNLF) eaders assisted in maintaining environmental protection of marine sanctuaries, even involving lawless elements (pirates) and the Abu Sayaf.  

Project implementation was slow. Initially there was an impasse between Department of Agrarian Reform (DAR) and IFAD on the role of NGOs in project management that took 18 months to resolve. In 2003, DAR conducted a comprehensive review of progress and processes and identified implementation problems and measures to address them. In late 2003, WMCIP was mainstreamed into regular activities of DAR Region IX. Withdrawal application processes were slow, resulting in project implementation delays. This seriously affected operations in 2006, and particularly infrastructure projects. DAR and the Project Management Office (PMO) have developed an exit strategy plan designed to ensure sustainability of activities after the WMCIP project is closed.

The project area is part of a conflict zone, which together with the widely scattered location of project sites, presented problems for implementation and supervision. This was not conducive for efficient and effective project management and supervision, and must be recognized as a constraint in operating in such areas. 

Relevance. The project is very relevant to the needs of the beneficiaries and to IFAD's strategic objectives and targeting in the Philippines. It is in line with the priorities of the Government in meeting the basic needs of the poor as embodied in the MTPDP and the Social Reform Agenda, the framework for poverty alleviation. However, the credit program had deficiencies, while the targeting of three different beneficiary groups added to management difficulties of an already complex project – the credit and coastal communities' components could have been excluded.

While the project was targeted at poverty reduction in one of the poorest parts of the Philippines, it recognized that increasing inequality and persistent poverty are concerns for future stability and is thus highly relevant to the needs of conflict-affected communities. It addressed two key objectives of IFAD's crisis prevention and recovery policy: (i) a proactive approach to addressing deep-rooted causes, such as land security and access to resources, services and opportunities; and (ii) a focus on institutional development at the rural level, enhancing local ability to respond to shocks resulting from civil strife and conflict.

Effectiveness and efficiency. The project goal was accomplished, nearly 22,000 households benefiting by June 2007. Physical accomplishments exceeded targets. Incomes increased, but generally remain below the poverty threshold. The small enterprise development and credit component, however, was not very effective. Resource use on most components was good, with almost all loan funds likely to be utilised before project closing, but the three year project extension enabled this.

Performance of partners. IFAD, UNOPS and the government and its agencies performed satisfactorily. IFAD was constrained by having no field presence, and supervision was handled by UNOPS, but IFAD staff should have participated in the Mid-term Review (MTR) field mission. UNOPS fielded well qualified supervision missions and their reports addressed major concerns and provided clear recommendations for actions, with respective responsibilities.

However, for the first half of the project life, UNOPS project personnel changed - there was little consistency of knowledge on project progress during this period, and no follow up on recommendations between missions. The government and its partner agencies have complied with loan covenants and implemented most recommendations of the supervision missions.  Initially project implementation progress was slow, but improved after the project was mainstreamed in DAR in late 2003.

Mainstreaming has raised the likelihood of sustainability. The performance of different NGOs was not consistent, with varying levels of capabilities and experience.  LBP and the Local Participating Credit Institution (LPCI) fulfilled their responsibilities, but LBP could have been more proactive in addressing credit design weaknesses.

Project impacts

Poverty impacts. Overall the project's impact on rural poverty was satisfactory, with improvements in physical and human assets, social capital and empowerment. Agricultural productivity and food security improved. The project had positive impacts on environment, but limited impact on creation of financial assets and in marketing. The project contributed to institutional strengthening at local levels and had a satisfactory impact on policy advocacy.

Sustainability and ownership. Sustainability of WMCIP community-initiated projects and activities is dependent on the capabilities of individual beneficiaries, barangay communities, POs and cooperatives. Capacity building takes time, and will need the continued support of LGUs and other agencies. Mainstreaming activities into regular provincial and regional programs, and continued provision of support activities, is essential for sustainability. DAR has already mainstreamed many activities into its own operations and has indicated it will continue to provide such support and act as the coordinating agency for other government agencies. Partner agencies have also indicated their commitment to the mainstreaming arrangements set out in the WMCIP exit strategy. WMCIP is thus potentially sustainable, with DAR taking a lead proactive role. However, maintenance of some of the rural infrastructure, and particularly farm to market roads, is beyond the capabilities of barangays and will require LGU technical and financial help. Many of the collective enterprises and other agri-business enterprises may not be sustainable due to the absence of effective advisory services. Neither the LGUs nor NGOs have the capability to provide this.

The participatory planning and social empowerment processes of WMCIP were critical in generating ownership of project activities by communities and beneficiaries, which will help in sustainability. DAR and some LGUs, such as the Zamboanga del Norte Provincial LGU, have taken a very active involvement and appear committed to continue their support.

Innovation, replication and scaling up. WMCIP adapted proven procedures for the Community and Institutional Development (CID) component; innovative NRM technologies were adopted, some of which have already been scaled up. But to enhance replication and scaling-up, agricultural production should move beyond subsistence farming practices and be more entrepreneurial – most of the livelihood projects have been on a backyard-scale, with few incentives for innovation. The project was not able to implement a replicable agri-business and market-oriented strategy.

Conclusions and recommendations

Performance Ratings of the WMCIP Project

Evaluation criteria

Evaluation ratings

Project performance

 

Relevance

5

Effectiveness

4

Efficiency

3

Overall project performance

4

Rural poverty impact

 

Physical assets

4

Food security

4

Agriculture productivity

5

Environment and natural resources

5

Human assets

5

Financial assets

3

Social capital and empowerment

5

Institutions and services

4

Markets

3

Overall rural poverty impact

4

Other performance criteria

 

Sustainability and ownership

4

Innovation, replication, scaling up

4

Performance of partners

 

IFAD

4

UNOPS

4

NGOs

3

Government and its agencies

4

Overall project achievement

4

Source: IFAD Evaluation Mission 2007

Overall performance of the WMCIP was highly relevant, targetting the poor, and was successful in meeting its targets, although implementation was slow. The project targeted the poorest in 81 barangays in the four provinces; inclusion from 2003 of vulnerable households significantly enhanced outreach and distribution of benefits. Incomes increased, and significant changes at the household level are evident, although poverty remains prevalent. Crop and fisheries production has led to diversification that has improved food security and nutrition intake. Capacity building has been substantial at barangay and LGU levels, with partnerships developed for supporting development activities. The small enterprise development and credit component was not successful. 

WMCIP's focus on indigenous people, and the inclusion of the small grant component - Support Project for the Indigenous Cultural Communities MNLF in the Zone of Peace within the Agrarian Reform Communities (SPICCnZPARC) that addressed concerns of the marginalized ex-combatants, highlights the depth of WMCIP engagement. Most barangays included are within conflict areas (leftist and Islamic groups). It was reported to the evaluation team that WMCIP was "brave" to have worked in these areas and in developing partnerships with its varied stakeholders. WMCIP clearly focused on the Millennium Development Goals of poverty and hunger, gender and equality and empowerment of women, and environmental sustainability. It addressed IFAD's overarching goal of enabling the rural poor to overcome poverty.

While WMCIP has been partly successful, much remains to be accomplished, with some infrastructure and livelihood initiatives still under implementation. The level of poverty at project commencement was high, and while incomes have increased, poverty remains below the threshold in many barangays. However, initiatives are in place in both natural resources and small enterprise development that provide a basis for further development to help increase incomes and improve livelihoods. Capacity at barangay and LGU level has been improved. Agency sustainability mechanisms have been established to help take the WMCIP initiatives further. A follow-on program is needed to take advantage of, and build on this.

Lessons learned and issues arising

The evaluation identifies several significant lessons/issues which have a bearing both on the future of WMCIP, and similar projects that might be pursued. Some of these relate to project design, and particularly the context within which the design is developed, others to project management and implementation, while specific issues have arisen over the enterprise development and credit component. These have been taken into account in the recommendation and sub-recommendations set out below.

Recommendations

IFAD activities should continue to support development in upland areas where poverty remains persistent and IFAD has experience. In particular, it will be desirable to continue working in the WMCIP upland areas of Zamboanga Peninsula 1. This recommendation could be part of a future IFAD-funded project covering two or three other upland regions in the Philippines. Its objectives would be to strengthen ongoing WMCIP activities, address its weaknesses, and help ensure sustainability of benefits. Requirements of coastal communities are different, and thus should be handled under a different project to ensure the required developmental results of those involved in artisanal fisheries.

If IFAD and the Government subsequently decide to undertake a future project focusing on upland areas such as in WMCIP areas then, the following sub-recommendations should be taken into account. These are grouped under recommendation 1.1 clarity of design, recommendation 1.2 project organisation and management, and recommendation 1.3 specific project components and implementation.

Recommendation 1.1 - Clarity of Design

Integrate the principles of a watershed and landscape approach to Natural Resource Management (NRM). For this it is recommended that:

  • In order to promote better control and accountability over resource destructive activities and the flow of positive benefits between communities (e.g. less siltation and improved water quality) within the project area, future interventions should work in a more limited geographic area.  Future interventions should be limited to headwater areas incorporating the principles of a landscape approach (see next bullet) considering downstream effects, but limiting implementation or support to critical uplandreas.
  • Within the upland areas, targeting of project sites should be to the extent possible contiguous for better environmental benefits and incorporate the principles of a landscape approach, which integrates social, cultural, and environmental concerns with the management of the land area, but with special care taken of the possibilities of environmental disturbances beyond the control of the project.
  • A locus for intervention in terms of geographic coverage and beneficiary needs has to be clearly identified during design of a potential second phase - together with the corresponding institutional considerations for the development of improved monitoring and supervision and implementation support arrangements.

Specify more accurately the target groups. Aligned with the Government of the Philippines (GOP) development thrusts and directions, the project design should be in line with the IFAD targeting policy and clear on the poverty level of the targeted groups, and whether to include the enterprising poor and vulnerable groups. WMCIP had a selection guide for vulnerable households and during implementation these were integrated with the KALAHI (Linking Arms to Fight Poverty) program priorities of the National Anti-Poverty Commission at the barangay and municipal level. This approach was useful and should be considered in the design of future projects.

Improved integration of components. In WMCIP, the different project components had impacts on the effectiveness of succeeding components.  For example, technologies under Component 2 had a high rate of adoption of innovations, this being partly attributed to a high rate of awareness resulting from the social preparation initiatives under Component 1. However the links between components 2 and 3 were not as strong (i.e. poor Small Enterprise Development Component (SEDC)). Also, the integration was not consistent across all project areas.  As such, any future operation should build on and improve the implementation of the approach adopted in WMCIP to ensure improved integration and sequencing of components and activities.

Enhance the government's participation in the design process.  In line with the evolving operating model within IFAD, future project design should involve the country program management team (CPMT) and enhance the participation of government, in all levels, in order to improve country ownership, relevance, and partnership.

Recommendation 1.2 - Project organisation and management

Mainstreaming for sustainability. Activities should be mainstreamed into regular regional and provincial operations of all agencies and sustainability instituted from project onset1 . In this regard, clear coordination mechanisms between partner agencies should be established. NRM in particular cuts across institutional mandates of several agencies, and the project design and logical framework should be clear on inputs, activities and expected outputs and impact. To enhance project mainstreaming, coordination mechanisms between the IFAD and the GOP/Executing Agency should be in line with the institutional set ups negotiated and agreed in the project loan agreement, based on a transparent assessment of the needs of the project and the existing institutional capabilities. Clarity of responsibilities is also important if the project covers parts of Autonomous Region in Muslim Mindanao (ARMM) as well as Region 9 (e.g. in Basilan).

Project management in conflict zones. Project execution and supervision and implementation support mechanisms need to be flexible, given the constantly changing security circumstances in the region. For example, reliance on local agencies may be necessary. Project management staff must be able to work with and communicate across the varied different groups in conflict areas: at local levels, being indigenous to the area or of the same ethnic group would be advantageous.

Increased IFAD visibility. IFAD needs to make its presence felt more widely during project execution, for example, by ensuring that its policy priorities and declarations (e.g. related to indigenous people) remain areas of focus throughout the project life cycle and undertaking direct supervision and implementation support including participation of the field presence officer.  The continuation of direct supervision and the strengthening of the field presence officer are contingent on available resources allocated within the wider framework of IFAD activities related to field presence and direct supervision.

Screen community initiatives: New community infrastructure projects, while continuing to be selected in a participatory manner by communities, should also be screened by the project for technical and environmental feasibility. Project appraisal mechanisms to ensure objective review and approval of infrastructure projects should be established.

Recommendation 1.3 - Specific Project Components and Implementation

Resources and environment

  • Mindanao conflict and regulation of resource use. Control and development of the region's lands and natural resources has contributed to the Mindanao conflict, particularly in terms of the inequitable use/control of resources. WMCIP made initiatives in peace and development, such as peace zones formation in Basilan, peace process consultation between the government and a splinter local rebel group from the Communist Party of the Philippines, and some training in conflict sensitivity and peace building. Future projects must recognize and support the dynamics of tri-communities (Muslim, Christian and Indigenous Peoples) in conflict areas by bringing these partners together to resolve conflicts and manage natural resources.  This good practice from WMCIP should be continued as conflicts around natural resource use are intrinsically anchored in the diversity of ethnicity, religion and socio-economic and cultural knowledge, structures and practices.
  • Environment. The influx of mining activities within the four provinces poses a clear threat to the sustainability of WMCIP and needs to be kept under review. If there is no IFAD follow on intervention, as part of the mainstreaming, DAR, Department of Environment and Natural Resources (DENR), and relevant LGU should be involved in this review.
  • Indigenous Peoples (IPs) and Certificate of Ancestral Domain Claims (CADCs).  Based on the WMCIP experience working with three communities to Convert their CADCs to Certificate of Ancestral Domain Titles (CADTs)3 , there are two pressing issues that affect the concern for Indigenous People and should be incorporated into future activities; (i) financing of Ancestral Domain Sustainable Development and Protection Plans; and (ii) organizing other IP groups within the region to formally file their respective CADC where viable under Indigenous Peoples Rights Act.

Capacity Building

  • Community development (i) Financial support by Municipal Local Government Unit (MLGUs) should be continued for the CDVs to support POs and development work in barangays, in coordination with the Sangguniang Pambarangay (Barangay Council). The financial management capabilities of officers of People's Organizations, Farmers/Fishermen's Associations and Cooperatives should be further enhanced and include provisions for assessing the economic viability of proposed investment activities. In addition, assistance should be provided in establishing market linkages.
  • LGU capacity development. (i) Continue training and technical support to Municipal and Provincial LGU personnel in monitoring and evaluation; and (ii) Continue support to LGUs in assessing and updating of the Sustainable Barangay Development Plans responsive to the emerging needs of the barangays and for fund mobilization.
  • Line agency support and partnership. (i) Line agencies should continue providing technical support to community organizations in pursuing NRM, livelihood and marketing and credit; (ii) Linkage of ongoing and new programs using existing structures such as Barangay Development Team/ Municipal Development Team and Barangay Infrastructure Monitoring Board should be pursued to ensure continuity of institutional development (and avoid duplication) in the identification and implementation of projects funded by other agencies.

Enterprise Development and Credit

  • Market-oriented approach. An integrated approach is needed covering production, processing and marketing, recognizing the importance of market linkages for the rural poor. Capacity-building and investment is needed in activities that are commercially viable in the market. NGOs may not have capabilities in enterprise development and business development services, and if used, need training.
  • Credit. A different credit modality should be sought with other government entities. This should take into account lessons learned from the evaluation of the previously IFAD-funded Rural Micro-Enterprise Finance Program and the recently launched Rural Micro-Enterprise Promotion Project (RuMEPP).  For example, RuMEPP's effort to use the credit funds as a deposit/guaranty in the Small Business Guaranty and Finance Corporation working in partnership with Micro-finance Institutions (MFIs) is a step in the right direction.
    If IFAD and the Government subsequently decide to undertake a future project in coastal areas then the recommendations under 2.1 coastal areas should be taken into account.

Recommendation 2.1 – Coastal areas – These recommendation are only relevant if there is a future intervention related to coastal issues.

Environment. If a follow on intervention continues to work in coastal areas, greater effort has to be made to enhance the involvement of the DA- Regional Field units and Bureau of Fisheries and Aquatic Resources, especially in regard to extending technical assistance to the various land and water resource management technologies.

Department of Environment and Natural Resources (DENR) – complementarity of two laws, namely RA 7586 and RA 8550, affecting marine and coastal resource management and fisheries in National Integrated Protected Areas Systems (NIPAS), needs to be addressed. The Fisheries Code (RA8550) is more localized and operable at the Local Government Unit (LGU) level. NIPAS requires congressional approval across a vast stretch of protected areas.


1/ As this was an interim evaluation, a key question for the evaluation from the approach paper was whether or not a follow-up phase of the project should be pursued.  Thus, in addressing this question, the evaluation suggests the need for follow on activities.  The evaluation believes that there are opportunities to build on the stronger project activities and to help address some of its weaknesses in order to help ensure sustainability of benefits.  As with the CHARM project area, where IFAD has been involved for more than 20 years, the WMCIP upland areas are a challenging environment and a longer term perspective may be required to ensure impact and sustainability. 

2/ Specifically during design IFAD should consider: (a) the responsibilities between the regional directors and the project managers; (b) the role of other staff of the regional bureaus of the line departments vis-à-vis those who may need to be recruited on temporarily basis; and (c) how to deal with the issues around the implementation of convergence between different line departments (DA, DAR, DENR, etc). 

3/ See Table 1. The logical framework results chain from the PCR.

 

LANGUAGES: English

Western Mindanao Community Initiatives Project

Philippines  
October 2009

Poverty in the Philippines is predominantly rural and Western Mindanao is one of the poorest and least developed regions in the Philippines. The agriculture sector accounts for over a third of total employment but production is not keeping pace with population growth. Low labour productivity characterizes the sector and a large part operates at subsistence level and is vulnerable to year-to-year weather changes.

About six out of ten people in rural areas depend on agriculture for their livelihood. Fishing is an important sub-sector, mostly at subsistence level. The current Medium-Term Philippine Development Plan (MTPDP) identifies Mindanao as an area of strong agricultural potential.

The project area is in a conflict zone. The conflict can be traced to centuries of discrimination, as perceived by the Lumad (indigenous) and Muslim people, that has marginalized them in terms of social and economic development. While a peace agreement was signed between the Philippine government and the Moro National Liberation Front (MNLF) in 1996, episodes of conflict in 2000 and 2003 have been costly, with the displacement of nearly one and a half million people, including those in project areas.  Despite the armed conflict, tri-communities co-exist in Mindanao (Lumads, Muslim ethnic groups, and migrant settlers from Visayas and Luzon).

The project has four components: (i) Community and Institutional Development, (ii) Natural Resource Management, (iii) Small Enterprise Development and credit, and (iv) Project Implementation. Total project costs were estimated at US$18.15 million of which the International Fund for Agricultural Development (IFAD) was to finance US$15.54 million (85.6 per cent), including a grant of US$ 0.75 million.

The goal of WMCIP was increased subsistence, higher incomes, better standards of living and greater resilience of livelihood of up to 16,000 farm and fishing households. Twenty one municipalities with 80 barangays ("village" level administrative unit) were pre-identified for inclusion at design stage (later increased to 81). The project was appraised during 1996-97 and approved in 1998, with an implementation period of six years; it became effective in 1999.  The original loan closing date was extended from Dec 2005 till Dec 2006, and now Dec 2007, with project completion 30 June 2007.

This interim evaluation followed the Office of Evaluation's (OE) methodology for project evaluation in assessing performance and impact. Its objective was to develop recommendations for enhancing the design and implementation of new and ongoing IFAD-funded projects, and to facilitate IFAD management's decision on whether or not a follow-up phase of WMCIP should be financed by the Fund. The Evaluation Team visited three of the four provinces covered by the project, Zamboanga del Norte, Zamboanga del Sur, and Zamboanga Sibugay.

Visits to Basilan and Tawi Tawi were not possible due to security concerns.  At the time of the field work, the self assessment by the government, project completion report, and United Nations Office for Project Services (UNOPS) final supervision mission report were not available, hampering the evaluation.

Performance

Design. The project components were designed to emphasize participatory planning and implementation and covered: community development and institutional capacity building, natural resource management and enterprise development.

It covered three supposedly contiguous sites where the most disadvantaged populations could be targeted, comprising upland areas, agrarian reform lowland areas, coastal areas, and indigenous peoples. Barangay project sites, however, were not contiguous, their wide spread locations making project implementation and supervision difficult. In addition, interconnectivity of upland-lowland-coastal ecosystems was not captured. The design, however, did focus on key environmental concerns that impacted on people's livelihoods: (i) over-exploited land resources, farmed in a haphazard manner, resulting in degradation and loss of soil fertility, affecting production and incomes; and (ii) depleted fish stocks from over-fishing and use of destructive methods and practices, resulting in damage to marine and water resources.

A pilot scheme to address concerns of vulnerable households was introduced in 2003, and expanded in 2005. The credit sub-component of the project was not implemented for three years, the design not being suitable to the needs of the project beneficiaries given the stringent lending policies of the Land Bank of the Philippines'(LBP - the implementing agency), and reluctance of credit conduits to participate in the credit program. Changes recommended by studies commissioned by the government to address the problem were not implemented.

Overall performance. By June 2007, the project achieved or exceeded practically all quantitative targets, apart from some infrastructure provision. Initial project start-up was delayed pending resolution of project management issues (para.18 below), while performance up to mid 2004, when the project was originally due to be completed, was slow. The project has been extended by three years. Over 9,300 poor farmer households and almost 2,400 poor fishermen households have directly benefited, including nearly 3,400 vulnerable households. Nearly 9,000 households are involved in the process of enterprise development. The draft Project Completion Report (PCR) states that households' income increased by 50 per cent to 75 per cent over six years. However, based on a survey conducted by the project in 2007, average annual income of beneficiaries has increased by about 38 per cent since 2005. This was largely due to higher farm incomes, attributable to the beneficiaries' adoption of new agricultural technologies under the project.

Community and institutional development. Involved the mobilization and participation of the community in the identification of development needs and in the prioritization of interventions responsive to the needs of the community. Targets for assisting community organizations were exceeded, and several innovative practices introduced are now considered as good practice. Local Government Units' (LGU) capacity was strengthened and partnerships developed with line agencies. The objective of improved community capacity to plan programs and access funds for the communities' priority projects was achieved.

Natural resource management. In land resource management, over 9,000 farmers were trained in appropriate farming technologies; nearly 8,000 adopted the technologies, conserving approximately 2,405 hectares. Project activities focused on conservation or regeneration of natural resources, new farming technologies reducing production costs and integrating short-term production (crop and animal) and long gestation crops in farming systems. They addressed nutrition, food security, environment protection, and improved productivity. Technologies introduced have been socially and culturally acceptable; training was provided to improve para-technicians' capabilities in transferring technology to other farmers in the community. If more beneficiary barangays had been contiguous, then benefits from the new technologies may have spread more widely.

In marine and water resource management, over 2,300 fishermen were trained in appropriate technologies, such as coastal resource management with built-in environmental protection features. Mangroves were rehabilitated, and artificial coral reefs installed; 289 hectares of municipal waters have been delineated and declared as Marine Sanctuary. The Marine Protected Areas, artificial reefs, and mangrove rehabilitation had positive effects on the marine environment and fish populations.

Indigenous people. The project in partnership with the National Commission on Indigenous Peoples (NCIP) helped security of land tenure, facilitating the award of Certificates of Ancestral Domain Claims (CADCs), and conversion to title (CADT).

Rural infrastructure. Slippage and time over-runs of infrastructure sub-projects occurred due to delay in fund releases, and to LGUs' absorptive capacity. Many sub-projects have not been completed, with over 40 not started; 12 sub-projects will be cancelled. Maintenance is a concern for some of the roads, which are beyond barangay capabilities. Sub-projects were chosen through community consultation meetings as part of preparation of the barangay development plans; however they were not then specifically reviewed to determine their feasibility and environmental viability.

Small enterprise development.  Overall numerical targets were achieved, but there is no evidence that beneficiaries are consistently engaged in enterprises and making reasonable returns, nor that effective government and private research and advisory services for on- and off-farm enterprises were successfully established. The LGUs and Non Governmental Organisation (NGO)s lacked capacity in business advisory services, and interventions did not produce desired results; only a few enterprises have potential. Few beneficiaries availed of credit, with low repeat credit availment. The savings and credit approach, however, was a positive project achievement.  Efforts to redesign the credit component did not materialize. Credit operations moved forwards towards the end of the project, but the project no longer had resources to provide advisory support.

Gender equality and mainstreaming. The proactive approach to include women in barangay and livelihood activities was very evident, with women's associations being formed in all project sites. However, instead of balancing and harmonizing roles of men and women, the project approach resulted in a compartmentalized perspective of gender and development.

Conflict and peace building. Natural Resources Management (NRM) partner agencies, NGOs-People's Organizations (POs), civil society and the armed forces converged in efforts to achieve the project objectives. The armed forces cooperated with civil society in the protection of the Buluan Fish Sanctuary. Support was extended by "leftist" elements in Zamboanga del Norte. In Zamboanga del Sur and Zamboanga Sibugay, Moro Islamic Liberation Front-Moro National Liberation Front (MNLF) eaders assisted in maintaining environmental protection of marine sanctuaries, even involving lawless elements (pirates) and the Abu Sayaf.  

Project implementation was slow. Initially there was an impasse between Department of Agrarian Reform (DAR) and IFAD on the role of NGOs in project management that took 18 months to resolve. In 2003, DAR conducted a comprehensive review of progress and processes and identified implementation problems and measures to address them. In late 2003, WMCIP was mainstreamed into regular activities of DAR Region IX. Withdrawal application processes were slow, resulting in project implementation delays. This seriously affected operations in 2006, and particularly infrastructure projects. DAR and the Project Management Office (PMO) have developed an exit strategy plan designed to ensure sustainability of activities after the WMCIP project is closed.

The project area is part of a conflict zone, which together with the widely scattered location of project sites, presented problems for implementation and supervision. This was not conducive for efficient and effective project management and supervision, and must be recognized as a constraint in operating in such areas. 

Relevance. The project is very relevant to the needs of the beneficiaries and to IFAD's strategic objectives and targeting in the Philippines. It is in line with the priorities of the Government in meeting the basic needs of the poor as embodied in the MTPDP and the Social Reform Agenda, the framework for poverty alleviation. However, the credit program had deficiencies, while the targeting of three different beneficiary groups added to management difficulties of an already complex project – the credit and coastal communities' components could have been excluded.

While the project was targeted at poverty reduction in one of the poorest parts of the Philippines, it recognized that increasing inequality and persistent poverty are concerns for future stability and is thus highly relevant to the needs of conflict-affected communities. It addressed two key objectives of IFAD's crisis prevention and recovery policy: (i) a proactive approach to addressing deep-rooted causes, such as land security and access to resources, services and opportunities; and (ii) a focus on institutional development at the rural level, enhancing local ability to respond to shocks resulting from civil strife and conflict.

Effectiveness and efficiency. The project goal was accomplished, nearly 22,000 households benefiting by June 2007. Physical accomplishments exceeded targets. Incomes increased, but generally remain below the poverty threshold. The small enterprise development and credit component, however, was not very effective. Resource use on most components was good, with almost all loan funds likely to be utilised before project closing, but the three year project extension enabled this.

Performance of partners. IFAD, UNOPS and the government and its agencies performed satisfactorily. IFAD was constrained by having no field presence, and supervision was handled by UNOPS, but IFAD staff should have participated in the Mid-term Review (MTR) field mission. UNOPS fielded well qualified supervision missions and their reports addressed major concerns and provided clear recommendations for actions, with respective responsibilities.

However, for the first half of the project life, UNOPS project personnel changed - there was little consistency of knowledge on project progress during this period, and no follow up on recommendations between missions. The government and its partner agencies have complied with loan covenants and implemented most recommendations of the supervision missions.  Initially project implementation progress was slow, but improved after the project was mainstreamed in DAR in late 2003.

Mainstreaming has raised the likelihood of sustainability. The performance of different NGOs was not consistent, with varying levels of capabilities and experience.  LBP and the Local Participating Credit Institution (LPCI) fulfilled their responsibilities, but LBP could have been more proactive in addressing credit design weaknesses.

Project impacts

Poverty impacts. Overall the project's impact on rural poverty was satisfactory, with improvements in physical and human assets, social capital and empowerment. Agricultural productivity and food security improved. The project had positive impacts on environment, but limited impact on creation of financial assets and in marketing. The project contributed to institutional strengthening at local levels and had a satisfactory impact on policy advocacy.

Sustainability and ownership. Sustainability of WMCIP community-initiated projects and activities is dependent on the capabilities of individual beneficiaries, barangay communities, POs and cooperatives. Capacity building takes time, and will need the continued support of LGUs and other agencies. Mainstreaming activities into regular provincial and regional programs, and continued provision of support activities, is essential for sustainability. DAR has already mainstreamed many activities into its own operations and has indicated it will continue to provide such support and act as the coordinating agency for other government agencies. Partner agencies have also indicated their commitment to the mainstreaming arrangements set out in the WMCIP exit strategy. WMCIP is thus potentially sustainable, with DAR taking a lead proactive role. However, maintenance of some of the rural infrastructure, and particularly farm to market roads, is beyond the capabilities of barangays and will require LGU technical and financial help. Many of the collective enterprises and other agri-business enterprises may not be sustainable due to the absence of effective advisory services. Neither the LGUs nor NGOs have the capability to provide this.

The participatory planning and social empowerment processes of WMCIP were critical in generating ownership of project activities by communities and beneficiaries, which will help in sustainability. DAR and some LGUs, such as the Zamboanga del Norte Provincial LGU, have taken a very active involvement and appear committed to continue their support.

Innovation, replication and scaling up. WMCIP adapted proven procedures for the Community and Institutional Development (CID) component; innovative NRM technologies were adopted, some of which have already been scaled up. But to enhance replication and scaling-up, agricultural production should move beyond subsistence farming practices and be more entrepreneurial – most of the livelihood projects have been on a backyard-scale, with few incentives for innovation. The project was not able to implement a replicable agri-business and market-oriented strategy.

Conclusions and recommendations

Performance Ratings of the WMCIP Project

Evaluation criteria

Evaluation ratings

Project performance

 

Relevance

5

Effectiveness

4

Efficiency

3

Overall project performance

4

Rural poverty impact

 

Physical assets

4

Food security

4

Agriculture productivity

5

Environment and natural resources

5

Human assets

5

Financial assets

3

Social capital and empowerment

5

Institutions and services

4

Markets

3

Overall rural poverty impact

4

Other performance criteria

 

Sustainability and ownership

4

Innovation, replication, scaling up

4

Performance of partners

 

IFAD

4

UNOPS

4

NGOs

3

Government and its agencies

4

Overall project achievement

4

Source: IFAD Evaluation Mission 2007

Overall performance of the WMCIP was highly relevant, targetting the poor, and was successful in meeting its targets, although implementation was slow. The project targeted the poorest in 81 barangays in the four provinces; inclusion from 2003 of vulnerable households significantly enhanced outreach and distribution of benefits. Incomes increased, and significant changes at the household level are evident, although poverty remains prevalent. Crop and fisheries production has led to diversification that has improved food security and nutrition intake. Capacity building has been substantial at barangay and LGU levels, with partnerships developed for supporting development activities. The small enterprise development and credit component was not successful. 

WMCIP's focus on indigenous people, and the inclusion of the small grant component - Support Project for the Indigenous Cultural Communities MNLF in the Zone of Peace within the Agrarian Reform Communities (SPICCnZPARC) that addressed concerns of the marginalized ex-combatants, highlights the depth of WMCIP engagement. Most barangays included are within conflict areas (leftist and Islamic groups). It was reported to the evaluation team that WMCIP was "brave" to have worked in these areas and in developing partnerships with its varied stakeholders. WMCIP clearly focused on the Millennium Development Goals of poverty and hunger, gender and equality and empowerment of women, and environmental sustainability. It addressed IFAD's overarching goal of enabling the rural poor to overcome poverty.

While WMCIP has been partly successful, much remains to be accomplished, with some infrastructure and livelihood initiatives still under implementation. The level of poverty at project commencement was high, and while incomes have increased, poverty remains below the threshold in many barangays. However, initiatives are in place in both natural resources and small enterprise development that provide a basis for further development to help increase incomes and improve livelihoods. Capacity at barangay and LGU level has been improved. Agency sustainability mechanisms have been established to help take the WMCIP initiatives further. A follow-on program is needed to take advantage of, and build on this.

Lessons learned and issues arising

The evaluation identifies several significant lessons/issues which have a bearing both on the future of WMCIP, and similar projects that might be pursued. Some of these relate to project design, and particularly the context within which the design is developed, others to project management and implementation, while specific issues have arisen over the enterprise development and credit component. These have been taken into account in the recommendation and sub-recommendations set out below.

Recommendations

IFAD activities should continue to support development in upland areas where poverty remains persistent and IFAD has experience. In particular, it will be desirable to continue working in the WMCIP upland areas of Zamboanga Peninsula 1. This recommendation could be part of a future IFAD-funded project covering two or three other upland regions in the Philippines. Its objectives would be to strengthen ongoing WMCIP activities, address its weaknesses, and help ensure sustainability of benefits. Requirements of coastal communities are different, and thus should be handled under a different project to ensure the required developmental results of those involved in artisanal fisheries.

If IFAD and the Government subsequently decide to undertake a future project focusing on upland areas such as in WMCIP areas then, the following sub-recommendations should be taken into account. These are grouped under recommendation 1.1 clarity of design, recommendation 1.2 project organisation and management, and recommendation 1.3 specific project components and implementation.

Recommendation 1.1 - Clarity of Design

Integrate the principles of a watershed and landscape approach to Natural Resource Management (NRM). For this it is recommended that:

  • In order to promote better control and accountability over resource destructive activities and the flow of positive benefits between communities (e.g. less siltation and improved water quality) within the project area, future interventions should work in a more limited geographic area.  Future interventions should be limited to headwater areas incorporating the principles of a landscape approach (see next bullet) considering downstream effects, but limiting implementation or support to critical uplandreas.
  • Within the upland areas, targeting of project sites should be to the extent possible contiguous for better environmental benefits and incorporate the principles of a landscape approach, which integrates social, cultural, and environmental concerns with the management of the land area, but with special care taken of the possibilities of environmental disturbances beyond the control of the project.
  • A locus for intervention in terms of geographic coverage and beneficiary needs has to be clearly identified during design of a potential second phase - together with the corresponding institutional considerations for the development of improved monitoring and supervision and implementation support arrangements.

Specify more accurately the target groups. Aligned with the Government of the Philippines (GOP) development thrusts and directions, the project design should be in line with the IFAD targeting policy and clear on the poverty level of the targeted groups, and whether to include the enterprising poor and vulnerable groups. WMCIP had a selection guide for vulnerable households and during implementation these were integrated with the KALAHI (Linking Arms to Fight Poverty) program priorities of the National Anti-Poverty Commission at the barangay and municipal level. This approach was useful and should be considered in the design of future projects.

Improved integration of components. In WMCIP, the different project components had impacts on the effectiveness of succeeding components.  For example, technologies under Component 2 had a high rate of adoption of innovations, this being partly attributed to a high rate of awareness resulting from the social preparation initiatives under Component 1. However the links between components 2 and 3 were not as strong (i.e. poor Small Enterprise Development Component (SEDC)). Also, the integration was not consistent across all project areas.  As such, any future operation should build on and improve the implementation of the approach adopted in WMCIP to ensure improved integration and sequencing of components and activities.

Enhance the government's participation in the design process.  In line with the evolving operating model within IFAD, future project design should involve the country program management team (CPMT) and enhance the participation of government, in all levels, in order to improve country ownership, relevance, and partnership.

Recommendation 1.2 - Project organisation and management

Mainstreaming for sustainability. Activities should be mainstreamed into regular regional and provincial operations of all agencies and sustainability instituted from project onset. In this regard, clear coordination mechanisms between partner agencies should be established. NRM in particular cuts across institutional mandates of several agencies, and the project design and logical framework should be clear on inputs, activities and expected outputs and impact. To enhance project mainstreaming, coordination mechanisms between the IFAD and the GOP/Executing Agency should be in line with the institutional set ups negotiated and agreed in the project loan agreement, based on a transparent assessment of the needs of the project and the existing institutional capabilities. Clarity of responsibilities is also important if the project covers parts of Autonomous Region in Muslim Mindanao (ARMM) as well as Region 9 (e.g. in Basilan).

Project management in conflict zones. Project execution and supervision and implementation support mechanisms need to be flexible, given the constantly changing security circumstances in the region. For example, reliance on local agencies may be necessary. Project management staff must be able to work with and communicate across the varied different groups in conflict areas: at local levels, being indigenous to the area or of the same ethnic group would be advantageous.

Increased IFAD visibility. IFAD needs to make its presence felt more widely during project execution, for example, by ensuring that its policy priorities and declarations (e.g. related to indigenous people) remain areas of focus throughout the project life cycle and undertaking direct supervision and implementation support including participation of the field presence officer.  The continuation of direct supervision and the strengthening of the field presence officer are contingent on available resources allocated within the wider framework of IFAD activities related to field presence and direct supervision.

Screen community initiatives: New community infrastructure projects, while continuing to be selected in a participatory manner by communities, should also be screened by the project for technical and environmental feasibility. Project appraisal mechanisms to ensure objective review and approval of infrastructure projects should be established.

Recommendation 1.3 - Specific Project Components and Implementation

Resources and environment

  • Mindanao conflict and regulation of resource use. Control and development of the region's lands and natural resources has contributed to the Mindanao conflict, particularly in terms of the inequitable use/control of resources. WMCIP made initiatives in peace and development, such as peace zones formation in Basilan, peace process consultation between the government and a splinter local rebel group from the Communist Party of the Philippines, and some training in conflict sensitivity and peace building. Future projects must recognize and support the dynamics of tri-communities (Muslim, Christian and Indigenous Peoples) in conflict areas by bringing these partners together to resolve conflicts and manage natural resources.  This good practice from WMCIP should be continued as conflicts around natural resource use are intrinsically anchored in the diversity of ethnicity, religion and socio-economic and cultural knowledge, structures and practices.
  • Environment. The influx of mining activities within the four provinces poses a clear threat to the sustainability of WMCIP and needs to be kept under review. If there is no IFAD follow on intervention, as part of the mainstreaming, DAR, Department of Environment and Natural Resources (DENR), and relevant LGU should be involved in this review.
  • Indigenous Peoples (IPs) and Certificate of Ancestral Domain Claims (CADCs).  Based on the WMCIP experience working with three communities to Convert their CADCs to Certificate of Ancestral Domain Titles (CADTs), there are two pressing issues that affect the concern for Indigenous People and should be incorporated into future activities; (i) financing of Ancestral Domain Sustainable Development and Protection Plans; and (ii) organizing other IP groups within the region to formally file their respective CADC where viable under Indigenous Peoples Rights Act.

Capacity Building

  • Community development (i) Financial support by Municipal Local Government Unit (MLGUs) should be continued for the CDVs to support POs and development work in barangays, in coordination with the Sangguniang Pambarangay (Barangay Council). The financial management capabilities of officers of People's Organizations, Farmers/Fishermen's Associations and Cooperatives should be further enhanced and include provisions for assessing the economic viability of proposed investment activities. In addition, assistance should be provided in establishing market linkages.
  • LGU capacity development. (i) Continue training and technical support to Municipal and Provincial LGU personnel in monitoring and evaluation; and (ii) Continue support to LGUs in assessing and updating of the Sustainable Barangay Development Plans responsive to the emerging needs of the barangays and for fund mobilization.
  • Line agency support and partnership. (i) Line agencies should continue providing technical support to community organizations in pursuing NRM, livelihood and marketing and credit; (ii) Linkage of ongoing and new programs using existing structures such as Barangay Development Team/ Municipal Development Team and Barangay Infrastructure Monitoring Board should be pursued to ensure continuity of institutional development (and avoid duplication) in the identification and implementation of projects funded by other agencies.

Enterprise Development and Credit

  • Market-oriented approach. An integrated approach is needed covering production, processing and marketing, recognizing the importance of market linkages for the rural poor. Capacity-building and investment is needed in activities that are commercially viable in the market. NGOs may not have capabilities in enterprise development and business development services, and if used, need training.
  • Credit. A different credit modality should be sought with other government entities. This should take into account lessons learned from the evaluation of the previously IFAD-funded Rural Micro-Enterprise Finance Program and the recently launched Rural Micro-Enterprise Promotion Project (RuMEPP).  For example, RuMEPP's effort to use the credit funds as a deposit/guaranty in the Small Business Guaranty and Finance Corporation working in partnership with Micro-finance Institutions (MFIs) is a step in the right direction.
    If IFAD and the Government subsequently decide to undertake a future project in coastal areas then the recommendations under 2.1 coastal areas should be taken into account.

Recommendation 2.1 – Coastal areas – These recommendation are only relevant if there is a future intervention related to coastal issues.

Environment. If a follow on intervention continues to work in coastal areas, greater effort has to be made to enhance the involvement of the DA- Regional Field units and Bureau of Fisheries and Aquatic Resources, especially in regard to extending technical assistance to the various land and water resource management technologies.

Department of Environment and Natural Resources (DENR) – complementarity of two laws, namely RA 7586 and RA 8550, affecting marine and coastal resource management and fisheries in National Integrated Protected Areas Systems (NIPAS), needs to be addressed. The Fisheries Code (RA8550) is more localized and operable at the Local Government Unit (LGU) level. NIPAS requires congressional approval across a vast stretch of protected areas.


1/ As this was an interim evaluation, a key question for the evaluation from the approach paper was whether or not a follow-up phase of the project should be pursued.  Thus, in addressing this question, the evaluation suggests the need for follow on activities.  The evaluation believes that there are opportunities to build on the stronger project activities and to help address some of its weaknesses in order to help ensure sustainability of benefits.  As with the CHARM project area, where IFAD has been involved for more than 20 years, the WMCIP upland areas are a challenging environment and a longer term perspective may be required to ensure impact and sustainability. 

2/ Specifically during design IFAD should consider: (a) the responsibilities between the regional directors and the project managers; (b) the role of other staff of the regional bureaus of the line departments vis-à-vis those who may need to be recruited on temporarily basis; and (c) how to deal with the issues around the implementation of convergence between different line departments (DA, DAR, DENR, etc). 

3/ See Table 1. The logical framework results chain from the PCR.

LANGUAGES: English

Rural Development Project for the Northeastern Provinces (PRODERNEA)

Argentina  
September 2009

In December 2007, the Executive Board of the International Fund for Agricultural Development (IFAD) requested the Office of Evaluation (OE) to undertake a completion evaluation of the Rural Development Project for the Northeastern Provinces of Argentina (PRODERNEA).

Evaluation objectives, methodology and process. Pursuant to IFAD's Evaluation Policy, the main objective of the evaluation is to assess project performance and impact, seeking at the same time to generate insights and recommendations for similar operations to take place in future in Argentina and elsewhere. The evaluation will also inform the country programme evaluation to be conducted by OE in 2009.

The evaluation was carried out in accordance with OE's Evaluation Manual. A preparatory mission took place from 2 to 6 June 2008, and the evaluation mission visited the country from 7 July to 1 August 2008. At the end of the field visit, an aide-memoire was presented in Buenos Aires introducing the main preliminary findings and salient points.

Socio-economic context and rural poverty. The period of more than 10 years encompassed by the PRODERNEA project cycle (from design in 1995 to closing in 2007) was a particularly complex and unstable one economically, socially and politically. The resulting environment was not favourable to rural development. In 2001 and 2002, the country underwent a severe economic and social crisis that slashed gross domestic product (GDP) by 20 per cent and brought significant setbacks to living conditions.

Since then, the Argentine economy has made a rapid recovery and is today one of the continent's most dynamic with annual growth of around 8 per cent. Agriculture plays a key part in Argentina's economy. In 2004, the sector contributed 58 per cent of total exports, employed 9 per cent of the active population and generated 9 per cent of GDP. Argentina is classified as an upper middle- income country under the World Bank's classification system, with gross national income (GNI) per capita in 2006 of USD 5,150.

The percentage of the population earning an income below the poverty line fell to 26.9 per cent in 2006, after having reached 48 per cent in 2003 and 57.7 per cent in October 2002.1 The incidence of poverty is much higher in rural areas, with marked differences among regions. The north-eastern and north-western provinces post the highest rates of rural poverty.

The project. PRODERNEA is the second phase of the Programme of Credit and Technical Support for Small Producers in Northeast Argentina (PNEA), completed in 1996. PNEA was a pioneering programme in a region – and a country – that had very little prior development experience focused on small-scale producers.

PRODERNEA has been implemented in the four provinces of north-eastern Argentina (Chaco, Corrientes, Formosa and Misiones). The target population identified in 1996 was 53,000 families, including 10,550 indigenous families. The overall objective was to contribute to overcoming the root causes of poverty in the north-east by strengthening the productive capacity of human and natural resources among poor people, small producers and indigenous people in the region, through a sustainable increase in incomes and self-management capacities. In order to achieve its objectives, the project included four main components: (i) technical services in support of production; (ii) financial services in support of production; (iii) an assistance fund for aboriginal communities (FACA); and (iv) project organization and administration.

The IFAD loan for USD 16.5 million was approved in April 1996, signed in September 1997 and declared effective in October 1998. The total cost of the project as originally designed was an estimated USD 36.4 million, including USD 8.3 million in cofinancing from the Inter-American Development Bank (IDB) and a counterpart contribution of USD 11.6 million. After the first four years of implementation delays, the Government of Argentina (GoA) and IFAD decided to reorient the project. An amendment was approved in December 2003 to introduce adjustments to current country conditions, which resulted in changes to: (i) the project term and scope; (ii) the strategy of some of the components; (iii) project coordination and administrative execution; and (iv) budget allocations for some components and categories of expenditure. The IDB cofinancing was cancelled and the total budget was reduced to USD 20.4 million. As of December 2007, 99 per cent of the IFAD loan had been disbursed. 

Performance and results

The original design of the project was relevant to the socio-economic context prevailing in Argentina in the mid-1990s, characterized by a predominantly liberal vision with the State reserving a compensatory role. The project stressed the provision of production support services – technical assistance and supervised and subsidized credit – as essential levers of development for the target population (IFAD 1996). 2

Project design following the reorientation approved in 2003 was consistent with the new political outlook and with IFAD's strategy in Argentina as set forth in the country strategic opportunities programme (IFAD 2004): favouring a development model with greater social inclusion of beneficiaries, selecting high-potential production units, moving closer to markets and positioning beneficiaries within virtuous value chains. Indigenous peoples continued to be supported through social development measures and specific instruments, such as subsidies rather than credit and greater coverage of training activities. On the other hand, the design did not assign sufficient importance to two key issues: (i) the land tenure situation; and (ii) the region's ecological conditions.

PRODERNEA's decentralizing approach is consistent with the principles underlying Argentina's constitution, and is widely recognized as having been successful. The provincial governments took on financial responsibilities (subsidiary loan agreements) as well as operational ones (administration and implementation), and developed a sense of relevance around the project. However, the complex design – involving five implementing units, four provincial and one national – led to initial delays in implementation, slow incorporation of provinces, cost increases and rescheduling of deadlines.

In terms of effectiveness, the project overall achieved most of its objectives, albeit with some limitations in scope and with varying results on meeting quantitative targets by component.3 As to credit, although the amounts granted exceeded post-reorientation targets, the approach adopted by PRODERNEA did not contribute to ensure access of the rural poor to rural financial services, but rather provided loans to a relatively small group of people for a relatively short period of time. The objective of institutional sustainability was not achieved, since the credit fund was not institutionalized in a way that would have enabled it to operate beyond the confines of the project and after closing. The lack of a rural finance policy places a major constraint on financial services development in Argentina. In addition, the relative delay in this area as compared to other countries in the region compromises the sector's competitiveness. By means of the production support services, affordable technologies were disseminated that proved to be consistent with the nature of small-scale production and had a largely positive impact on employment.

Services provided focused on production issues and, to a lesser extent, commercialization, despite considerable challenges in this area. The engagement of private partners for the provision of technical assistance is very limited.

Several factors affected efficiency by increasing the share of administrative costs to the detriment of resources directed to beneficiaries. These included the characteristics of the region and the geographical dispersion of beneficiaries, initial delays in implementation – which led to postponing project completion from June 2004 to June 2007 – and the structure of the implementing agency, with one central unit and four provincial units.

The project had a positive impact on improving the incomes and assets of the family farmers assisted, as well as their food security as a result of increased production. There is also a demonstrated improvement in the living conditions of aboriginal communities benefiting from the project, including basic services such as electricity and water as well as food security, although the latter continues to be precarious. In addition, despite shortcomings in participation mechanisms, beneficiaries continued to play a leading role in identifying projects, and the project's social acceptance rate was high.

On the other hand, project implementation did not make a positive contribution either to resource conservation and the environment generally, or to developing an environmental awareness to create conditions for future demand of sustainable development policies.

The component in support of indigenous peoples is regarded – in the context of public intervention in the north-eastern provinces – as a unique and valuable differentiation targeted to highly vulnerable social groups in rural areas, offering an alternative to the traditional welfare-based approach. The results have been positive, mainly in social capital, though also in the visibility and recognition of indigenous peoples as economic agents and interlocutors for provincial governments. However, insufficient consideration was given during formulation to the critical level of poverty – in terms of unmet basic needs – evident in the communities. Also, the lower relative importance assigned to this project component, together with the limited capacity of implementing units, resulted in insufficient attention being given to indigenous cultural issues. No actions were taken to influence public policies directed towards indigenous peoples.

The project promoted a gender approach to its activities overall and trained technical and implementation teams to build this dimension into project management. Nevertheless, the percentage of women, as direct beneficiaries and as loan recipients, is low. In addition, the inclusion of young people (not provided for in the original project design) came about only after reorientation, along with recognition of the importance and need to identify a strategy and activities to address this crucial segment of the population, particularly in less favoured rural areas. This objective was achieved only in part, through training, a young entrepreneurs project competition in Chaco province, and a course to train local development agents in Corrientes.

The project planned an exit strategy, reflecting its concern and commitment to ensuring continuity following the conclusion of external support. Among the achievements of this strategy, and also important pillars of sustainability, are the institutionalization of the policies by the Secretariat of Agriculture, Fisheries and Food (SAGPyA), new rural development institutions within provincial governments, and advanced negotiations on a new IFAD loan for the National Rural Areas Development Programme (PRODEAR) in the country's north-eastern and other provinces. On the other hand, staff and budget resources are limited among the institutions involved post-completion, and it is unlikely that the credit fund will be sustainable.

Innovative experiences for Argentina promoted by the project included provincial management combined with national coordination through the national coordinating unit within SAGPyA. In addition, new instruments were adopted, including liaison with regional producers' organizations and linkages with commercial value chains. The project promoted public-private partnerships to provide services to beneficiaries in organizing crafts production and commercialization, commercializing livestock production (livestock fairs) and farming insurance for market gardeners.

Also of note are PRODERNEA's contribution to and promotional role within the MERCOSUR Commission on Family Farming (REAF) as an innovative element, in respect to both IFAD's programme in the region and other SAGPyA-supported projects in Argentina.

IFAD and the GoA showed a high degree of commitment, flexibility and responsiveness in introducing needed changes during the 2002 reorientation. Nevertheless, during the design stage, IFAD was not able to ensure the necessary level of local participation and did not give sufficient consideration to lessons learned from the previous project. In addition, the Government's performance during the first phase was lacklustre, exacerbated by poor relations with the provinces, but improved substantially during the second phase. The Andean Development Corporation (CAF) covered loan administration satisfactorily.

Rating Summary

Evaluation Criteria

Ratings

Core performance criteria

 

Relevance

4

Effectiveness

4

Efficiency

4

Project performancea

4

Rural poverty impact

5

Household income and assets

5

Human and social capital and empowerment

4

Food security and agricultural productivity

5

Natural resources and the environment

3

Institutions and policies

6

Other performance criteria

 

Sustainability

4

Innovation, replication and scaling up

5

Overall project achievement b

4

 

 

Performance of partners

 

IFAD

4

Government

4

CAF

5

a/ Average of ratings for relevance, effectiveness and efficiency.

b/ The overall project achievement rating is calculated based on the ratings for project performance, rural poverty impact, sustainability and innovation.

Rating scale:  6 = Highly satisfactory; 5 = Satisfactory; 4 = Moderately satisfactory; 3 = Moderately unsatisfactory; 2 = Unsatisfactory; 1 = Highly unsatisfactory.

 

Conclusions

PRODERNEA, taking up efforts begun in the north-east by PNEA (the first programme addressed to small farmers in the region), was strongly marked by an extremely variable social, economic and political context, including different development approaches as well as highly diverse national and sectoral policies, which generated an unfavourable environment for rural development. During the project cycle, four different administrations came to power in the Executive Branch, and the country temporarily lost 20 per cent of GDP.

Nevertheless, the project as a whole achieved most of its objectives and – albeit with certain limitations – executed all of the funding allocated under the reorientation, and generated a series of complementary outputs and positive externalities.

The reorientation initiated in 2003 showed sufficient flexibility and responsiveness in modifying areas that were not performing well in a new socio-economic and public policy framework. In this new context, the project evolved and contributed to establishing a more complex and comprehensive vision of rural development, which extended beyond the agricultural sector to encompass all existing actors and rural-urban linkages in a given territory, highlighting the importance of producers' organizations and social and production networks.

Reorientation reinvigorated implementation under a more socially and economically inclusive approach, promoting beneficiary participation in all project actions and helping to incorporate production units into commercial value chains.

Beyond the results obtained – which are limited by its modest scale of investment in a country as large as Argentina – the project stands out for its contribution to mobilizing social, financial and physical assets and leveraging investment by the public and private sectors. PRODERNEA was successful and innovative in combining the implementation of direct project components with promotional activities to strengthen the visibility of the small-scale family farming sector – in a country characterized by an extraordinarily entrepreneurial agricultural sector – and support for formulating specific policies reflecting the importance of family farming at the national level.

This activity, centred on actions by the national coordination unit, was carried out in several ways: encouraging national debate on related issues, supporting activities in connection with Argentina's participation in REAF, and helping to promote the movement known as the National Family Farmers Forum. The latter two activities have taken shape as of 2004. One important impact to which this effort contributed is the creation – in March 2008 – of the Under-Secretariat for Rural Development and Family Farming within SAGPyA.

In addition, PRODERNEA successfully introduced and consolidated innovative institutional proposals for Argentina by improving the climate for relations between the Nation and the provinces, which initially did not favour concerted action. The project's inclusion within provincial government structures as called for in the design was highly positive in improving local ownership and future sustainability, thus building capacity nationally and above all at the provincial level to implement rural development programmes. Innovations with respect to regional liaison, value chain linkages and public-private efforts are highlighted.

The project carried out an important effort in identifying good practice and distilling lessons, and it has generated a notable volume of participatory evaluations. The project has provided an opportunity for collective reflection with all actors, helping to create a culture of dialogue and learning among regional participants, and has served as a platform for important policy dialogue initiatives.

Recommendations

The evaluation puts forward the following recommendations:

Negotiate a framework agreement at the national level, within which specific projects will be negotiated with each jurisdiction. In large countries with federal constitutional structures such as Argentina, additional review is needed for any future project proposals calling for decentralized implementation in the provinces. More in-depth consideration needs to be given to the impact of gradually incorporating the provinces over time, as naturally occurs, and to the specificities and autonomies involved in different administrative and political jurisdictions. Each of the projects under the framework agreement would be negotiated with the provincial authorities accompanied by explicit statements of political intent to implement them by stakeholders. Also, operating regulations – beyond general guidelines – should be established in the course of each specific negotiation process.

Strengthen social capital through partnerships among various economic actors in rural development, as a strategic thrust for development policies and projects. It is important to transcend the bounds of family farming to encompass the development of all relevant territorial actors. The following elements should be integrated in rural development: supporting the consolidation of existing local and regional organizations; linking producers and the entire rural population with virtuous commercial and industrial value chains; linking producers with all public and private services providing support for production and a better quality of life for rural society.

Promoting dialogue, research and design of sound rural finance policies in Argentina. Some processes under way such as the policy discussion on rural finance around REAF initiatives provide an opportunity for dialogue in this regard. IFAD, in particular, should support such discussion and policy-making processes, identifying success factors in other countries and facilitating exchanges with other projects. In addition, important lessons can be drawn from successful experiences in the country, such as the Social Capital Fund (FONCAP) and the examples of cooperatives receiving funding from PRODERNEA in Misiones.

Strengthen the rural technical assistance services system able to provide holistic responses to the producers' demands. To this end it is recommended, first of all, expanding the range of technical services -beyond the current concentration on aspects of production- to include multi-disciplinary teams with experience in areas such as marketing, commercialization and organizational strengthening, making sure the continuity of the technical assistance is guaranteed throughout the process. Second, it is necessary to support initiatives to develop or strengthen inter-institutional partnerships with public and private organizations, such as for example the National Institute of Agricultural Technology (INTA) and others. Third, it is important to promote the development of institutional frameworks that promote cooperative contracting of private technicians, with the collaboration of public authorities when necessary, particularly at the project organization and start-up stages.

Design and implement differentiated and specialized projects to improve living conditions for the indigenous population. These projects should be independent of those targeted to commercially-oriented family farmers, leading to effective affirmative action. Such projects should be designed and implemented by multidisciplinary technical teams trained to work with indigenous people, in participatory initiatives under the leadership of social actors that focus on improving the lives of the target groups. It is also necessary a consistent institutional framework that provides for advocating and developing policies that meet the needs of beneficiaries.

Environmental sustainability should play a central role in rural development strategy. The challenge associated to a larger pressure on natural resources (water, soil, vegetation) as a consequence of the expansion of the agricultural frontier, more intensive production methods, and a limited environmental awareness is a key issue that needs to be addressed beyond the possibilities of individual projects. This situation calls for policy dialogue at local, provincial and national levels focusing on sustainability.


1/ UNDP (2007): Argentina: Millennium Development Goals. Country Report.

2/ This comment does not apply to indigenous people, for whom a differentiated strategy was developed.

3/ The target number of beneficiaries under technical assistance and credit was not met. However, the targets set for the amount of credit and a number of projects under FACA were exceeded.



 

LANGUAGES: English, Spanish

Democratic Peoples’ Republic of Korea: Uplands Food Security Project (UFSP)

Korea, D.P.R.  
May 2009

Interim evaluation

Introduction

In April 2008, IFAD's Office of Evaluation (OE) conducted an interim evaluation of the IFAD-funded Uplands Food Security Project (UFSP) in the Democratic People's Republic of Korea (DPRK). This was the first evaluation by OE in the country. Its main objectives were to: (i) assess the performance and impact of the project; and (ii) generate a series of findings and recommendations for future projects and programmes financed by IFAD in the country.

On 26 November 2008, a learning workshop was held in Pyongyang to discuss the main results of the evaluation, and to provide inputs for the preparation of this agreement at completion point (ACP). The latter presents an agreement between IFAD (represented by the Asia and the Pacific Division) and the Government of DPRK (represented by the Ministry of Agriculture) on the key evaluation findings and recommendations, and proposals on how to act upon them.

Main evaluation findings

Main results per component. Of the seven project components, the sustainable crop production systems component was the most significant, with 56 percent of the original base cost in favour of 37 cooperative farms (CFs). It promoted the introduction of three-year crop rotations, albeit without the broad-based insertion of fodder crops. The component also included the procurement of farm machinery and farm inputs, mostly fertilizer. The second component, potato seed supply development, was successful as it responded to a key limitation of potato production, i.e. disease-free quality seed. Under the third component, environment preservation, the project encouraged 46 CFs to plant and enrich fuel wood plots on 5,500 hectares of nearby sloping lands.1

Credits for CFs and individual households were the object of the fourth component. Both types of borrowers took up small livestock breeding and fattening activities thanks to available credit, with a ceiling of less than US$70 per household. The target of reaching close to 18,000 households was achieved, covering 46 CFs. The fifth and sixth components, both very small in terms of financing, aimed at strengthening community facilities and services in 37 CFs. Both were implemented rather late, but still resulted in the construction of 162 community facilities such as kindergartens, nurseries, clinics and training centres and the procurement of more than 100 items of farm output processing equipment, which are now in operation and range from flour and rice mills to oil and potato starch extractors.

Under the seventh component, the project management unit (PMU) was strengthened, enabling it to assure project implementation. 

Project performance. The project was relevant in that it addressed significant rural poverty in relatively remote and resource poor areas of the country. Its design drew lessons from former IFAD operations in the country, especially with regard to the credit component. However, relevance was hampered by substantial information gaps at design and a deficient involvement of local expertise.

Thus, project design was unclear regarding the cropping systems to be promoted under the main component, and did not take into account the pervasive soil acidity in most of the participating CFs.

As it turned out, crop response to fertilizer applications was constrained because of this phenomenon, which the project did not address. Moreover, project design ignored the communication problems between the PMU and project partners, including IFAD, and failed to formalize co-financing and technical assistance agreements. As a result, neither food-for-work from the World Food Programme (WFP) nor technical assistance from the Food and Agriculture Organization (FAO) and the United Nations Development Programme (UNDP) were delivered as envisaged.

During several years the project had very limited success in introducing sound crop rotations because of inadequate approaches and technical advice. However, after a less directive approach was adopted favouring capacity-building in collaboration with the National Academy for Agricultural Sciences (NAAS), the project was effective in introducing sound crop rotations on more than 31,000 hectares against a target of 18,000 hectares. The project also succeeded in providing farm machinery, farm inputs, community facilities and services, and the farm output processing equipment. It also put in place the credit component in favour of 46 CFs and 18,000 individual households. In terms of efficiency, the goods were procured at comparatively competitive prices. However, end-user efficiency was constrained due to bottlenecks in maintaining farm machinery and the less than satisfactory crop response to fertiliser application because of widespread soil acidity. The credit component suffered an important efficiency loss.

Rural poverty impact. The impacts achieved in the domains of household assets and income, social capital and empowerment as well as agricultural productivity and food security were satisfactory, with the credit component having played a pivotal role. Over project life, household incomes increased by a factor of two in average, while livestock income – a direct consequence of the credit component – tended to grow more rapidly than farm dividends. The credit component empowered women in particular, as more than 90 percent of the loans were underwritten by women, and talent was unleashed to run small scale livestock operations at own risk and responsibility. This has contributed to enhanced food security. The impacts regarding natural resources and the environment were respectable in the immediate vicinity of the CFs, but unavoidably limited when considering the degree of degradation of sloping land on a broader scale.

Sustainability. The project has invested in areas with a distinct potential for sustainability, such as new crop rotations, the technically sound production of disease free potato seeds, and small livestock breeding and fattening systems, with CF work teams and households working in tandem. Increased household incomes and assets are likely to protect the households from future shock and stress. However, farm machinery procured by the project is losing its usefulness due to difficult access to spare parts and lack of adequate machinery maintenance capacity. The overall resource base of the CFs is still fragile, and the challenge of producing enough food for the country's population remains substantial. The encroaching of sloping lands is a matter of environmental concern, which the Government expects to resolve in cooperation with international partners.

Innovation. The project was successful in devising a low-cost and zero-risk credit component, which can be considered innovative. There is no evidence, however, of replication or scaling up of the household credit scheme. Although the new crop rotations took time to be adopted they may be the expression of local expertise and resolve rather than project-driven innovation. In matters of comprehensive soil fertility management, the project failed to take advantage of available local knowledge as no specific activities, such as systematic on-farm trials, had been included in the design.

Performance of partners. IFAD deserves the credit for having designed a relevant project in rural poverty reduction. On the other hand, important aspects, such as on-site fact-finding, the formation of solid partnerships with co-financiers and the securing of reliable communication links, were neglected both at design and during project implementation. The United Nations Office for Project Support (UNOPS) was in a similar position, confronted with the task of supervising a relatively complex project in an environment characterized by scarce availability of information and restricted access. While the performance of the PMU was respectable, considering all the odds, the other national partners remained reclusive in terms of information sharing and did not provide all relevant technical, economic and social data in time. The partners that exhibited outstanding performance were the beneficiaries, the CFs and involved households alike. The key potential for future success lies with the talent and determination of these men and women.

Recommendations

As this was an interim evaluation, the recommendations aim at setting a preliminary stage for the design of a subsequent IFAD operation in DPRK. They will be taken into account in the forthcoming results-based IFAD country strategic opportunities paper (COSOP) – a necessary step before initiating the design of a follow-up IFAD investment loan.

This table presents the evaluation recommendations 2 and the agreement status by IFAD and the Pacific and the Government of DPRK. In accordance with the IFAD Evaluation Policy, in case of disagreement, views are justified by both parties.


1/ Sloping lands are defined in DPRK as lands with slopes of 15 per cent or more. These lands are managed by the Ministry of Land and Environmental Protection

2/ Additions made to the interim evaluation recommendations, based upon the deliberations held during the learning workshop in Pyongyang on 26 November 2008 with the main project partners, are shown in italics

LANGUAGES: English