Ha Giang Development Project for Ethnic Minorities (2004)
Interim Evaluation 1Introduction and approach
The Mission was mounted to assess the performance and impact of the Project (HPM) in the context of consideration of a second phase of a new project for Viet Nam. The evaluation followed the guidelines and criteria of the IFAD Methodological Framework for Project Evaluation, with its emphasis on: rural poverty impact at household and community level; the Performance of the Project; and Performance of Partners.
Following preparatory work by OE in the provision of Approach and Issues Papers, the Mission fieldwork took place mostly in March 2004 under the aegis of the Ministry of Planning and Investment (MPI), and the Provincial and respective District Peoples Committees. Initial briefings were also held with FAO, SIDA, UNDP, UNICEF, and World Bank and the work of the Mission was greatly assisted by the active participation of Project staff and local officials. All ten districts were covered and visits made to 42 communes and 37 villages. More than 110 farmers and village people, including 42 women, were met and interviewed in one to one or in group discussions; and the IFAD evaluation matrix form was completed. Mission findings were presented in aide-memoirs at Wrap-up Meetings in Ha Giang and in Hanoi. At neither meeting was there any substantive disagreement or dissent as to the Mission findings and recommendations.
Development perspective. The economic transformation of Viet Nam since the early 1990s has been remarkable, with increasing liberalisation and private sector and foreign participation. Over the last five years, population growth and price inflation have both averaged less than 2% and debt service is now only 6% of GDP. Between 1992 and 1997, annual real growth in GDP consistently exceeded 8%; it is currently around 7%; and is planned - and forecast - to achieve 7.5% for the years 2000 to 2005. Despite this buoyancy, GDP per head remains at USD 460, so that Viet Nam remains one of the poorer countries, with poverty still widespread, especially in the densely-populated or remote rural areas, although prevalence nationally has fallen from 58% of the population in 1993, to 29% in 2002. However, poverty has marked spatial and demographic dimensions and the rate of decline of poverty incidence during the 1990s for ethnic minority groups was less than one third of that for the majority.
The agricultural sector recorded growth of over 4% per year during the 1990s, as a result of de-collectivisation and commercialisation. The change in agriculture from a predominantly peasant subsistence system to an industry that apart from providing the majority of domestic food needs is now number two in world rice and coffee exports, has been dramatic. Agriculture is estimated to employ some 70% of the workforce and to account for over 22% of GDP and about 30% of all exports.
In recent years, government policies have set out to change the development agenda and approach. Although there is no formal decentralisation policy, various decrees, laws and programmes have, since the early 1990s, had the effect of delegation of responsibility for development downstream and at the same time the emphasis on poverty alleviation has become explicit, with adoption of the Comprehensive Poverty Reduction and Growth Strategy (CPRGS) by government in 2002.
Design features and changes
Ha Giang is a large, sparsely populated province on the Chinese border with a total population of some 650 000 people or 110 000 households in 190 communes and 1 500 villages. Nearly 90% of its inhabitants belong to ethnic minority groups that have only recently started to enter mainstream society; and nearly half of its area comprises rocky, steep, mountainous terrain with harsh climatic conditions and limited natural resource potential. These characteristics prevail in the so-called Zone I, the northern border districts of Quan Ba, Yen Minh, Dong Van and Meo Vac; and to a degree in the north-western Zone II districts of Hoang Su Phi and Xin Man; the remaining southern, central and eastern Zone III districts of Bac Quang, Vi Xuyen, Bac Me and Ha Giang are more amenable, better serviced and economically more advanced.
Development in the province needs to be seen in the context of its frontier location and the dispersion and backwardness of its population. Ha Giang lies near the bottom of the national human index rankings, with an actual poverty incidence in remote communities of from 25 to over 50%. Provincial development policies and plans reflect the prime importance of infrastructure and pro-poor services. Less than 20% of the land is suitable for agriculture; about 40% is forest, with major steepness, encroachment and exploitative use problems. The prevailing total farm size is 1 to 1.5 ha, with less than 0.25 ha per family of terraced or watered land. Paddy, maize and root crops dominate the farming system -- but may not always be sufficient to meet household basic food needs -- and livestock for animal draught, as a financial reserve and a source of food and cash income are important. Customary rights of use and security of tenure for farmers are largely assured -- and are being reinforced by the ongoing allotment of land certificates.
The main elements of the rationale for HPM were the prevalence of poverty, poor health and illiteracy; the extreme difficulty of access, communication, service provision and marketing; the scarcity of arable land; the poor quality of the natural resource base and particularly the problem of potable water supply; and absence of opportunities for non-agricultural employment and income generation. Given these circumstances, the emphasis in design was on participatory local identification of priorities; local ownership of activities by implementation through existing agencies; flexibility in choice of sub-projects; and complementarity with provincial and other development operations. Hence, a strategy of direct action was adopted, to assure food security, enhance household incomes, improve access and communication, address the deficiencies of social and welfare status and inculcate and foster a local capacity for participatory planning and communal involvement in implementation.
The main changes in project design in implementation were: the amendment and curtailment of certain of the TA inputs when the project directorate had serious management problems with the inputs from the co-financier, SIDA; and, notably, the introduction of the Commune Development Fund, as a means of energising beneficiary interest and the flow of sub-project proposals.
The Project area was defined as all ten districts for the extension, animal health and income diversification activities, but only between four and nine districts for other components. In particular, the health and water supply activities were confined to the four northern and least advantaged districts. Much of the infrastructure investment was pre-targeted to commune and village level. At appraisal, targeting was proposed but no explicit criteria, nor were estimates for target group quantification provided. The target group was defined as the poor and poorest households, as per the national poverty line. This implied that the vast majority of households in the province belonged to the target group; almost entirely of ethnic groups; located in upland areas; and reliant on shifting cultivation within an increasingly diminishing land resource base; the ethnic minority population at the time was estimated at 500 000 people or 88 000 households, of whom 34 000 were considered as primary target group.
The overall goal of the Project was: to improve the living standards of the target group, that is mainly the ethnic minorities living in marginal upland areas. The specific objectives were: to raise household incomes and improve food security for the poorest households, while improving utilisation of natural resources; to support development of appropriate rural and social infrastructure; and to develop environmentally sustainable and culturally sensitive development models that can be replicated elsewhere in the northern provinces.
HMP had five components and eleven sub-components or activities, as summarised below:
Rural Infrastructure (Base cost USD 5.26m, 32% of total):
- Pedestrian Roads and Bridges - construction of key village paths, roads and bridges;
- Domestic Water Supply - construction/upgrading of village water supply schemes; and
- Irrigation - construction/upgrading of provincial (30+ha) and micro (<5ha) schemes.
Agriculture, Livestock and Forestry Development (Base cost USD 3.50 m, 21% of total):
- Farmer Extension - establishment of farmer-based extension systems and research support;
- Animal Health - improvement of animal health delivery systems; and
- Forest Protection - support for conservation models for critical watersheds/poor villages.
Income Diversification (Base cost USD 1.02m, 6% of total):
- Income Diversification - identification of opportunities, support/promotion, credit provision.
Social Development (Base cost USD 4.95m, 30% of total):
- Education - classrooms/centres, curricula/materials development; female literacy and training; and
- Health - training/establishment of VHWs, upgrading health centres, hospitals, free drug supply.
Project Management and Coordination (Base cost USD 1.9m, 11% of total):
- PCUs - establishment and support of Project Coordination Units at province and districts; and
- Monitoring and Evaluation - at province and district/department level.
Total Project costs, including contingencies, for the 6 year duration were USD 18.4m; the breakdown of financing was as follows: IFAD loan - USD 12.5m; UNDP grant - USD 2.3m; SIDA grant - USD 0.8m; government - USD 2.1m; and beneficiaries - USD 0.6m.
The implementation arrangements gave national responsibility to the Departments of Agricultural Economy and Foreign Economic Relations of MPI, which handled liaison with other Ministries. At the province level, accountability for progress and performance lay with the Provincial Peoples Committee (PPC), the Provincial Project Steering Committee (PSC); and the Provincial Project Coordination Unit (PPCU). The PPCU provided guidance, liaison and arrangement of technical support to: participating districts, for each of which there was a District Project Coordination Unit (DPCU); and to downstream agencies, principally district sectoral departments and their activity management teams. The PPCU was also responsible for liaison and dialogue with donors - IFAD, UNDP and SIDA - and associated agencies such as UNOPS, the Swedish Embassy and NGOs.
At the district level, District Peoples Committees (DPCs) were the prime movers in planning and implementation, backed up by the DPCUs. The commune is the link between villages and districts; the Commune Peoples Committee was involved in planning and oversight of Project interventions, mainly through the Commune Development Board (CDB), a special entity set up by the Project principally to manage the Commune Development Fund (CDF). Village Coordination Units (VCUs), also established by the Project, took on the role of assessing village development needs, communicating these to the CDB -- and passing on proposals and plans for sub-projects.
Summary results of implementation
Financial aspects of performance. The funds available as appraised were USD 18.4 million, equivalent, at that point, to some VND 233.9 billion. Expenditures to date are calculated to be VND 233.1 billion, that is 99.7% of the projected total in local currency terms, equivalent to some USD 16.3 million at the average USD/VND exchange rate for the Project period. In assessing these figures, it needs to be borne in mind that: the UNDP grant for TA was reduced from the original USD 2.3 million by USD 500 000 - or about VND 7 billion; the SIDA contribution amounted to only VND 4 billion, against an original target of VND 10 billion; and the Dong has depreciated against the US Dollar by over 30% during the Project period. The net result is that there remained at December 2003 outstanding amounts of VND 27 billion from IFAD funds and VND 2.2 billion from government funds to meet payments up to Project closure. Inclusion of these sums would bring total expenditure to VND 262.26 billion, or USD 18.34 million.
In comparing actual against target expenditures, there is the complicating factor of the CDF, which was not included as a cost component at appraisal but was financed from surplus funds from the low uptake of credit and social development. In the event, the main divergences by component have been: expenditures on Rural Infrastructure have been 132% of proposed, due to the heavy emphasis on infrastructure of CDF expenditures; Agricultural and Social Development and Income Diversification have reached only about two thirds of the values expected; and Project Management and Coordination costs have been over budget at 122%. In cost categories, significant divergences are: the low costs for Training and TA, at only 63% of planned, due to reduction of co-financier inputs; and very low recurrent costs, due to effective control and savings on plan, at only 45% of budget. It is clear from the financial analysis that the Project has enjoyed sound cost control and financial management.
Physical achievements. The key indicators of benefit of HMP are that commune-wise, the pattern of coverage is: roads and bridges - 20; water supply - 24; irrigation - 92; farmer extension - 132; animal health - 191; income diversification - 60; education - 70; health - 67; and the CDF 78. The Project has thus had a significant, multi-component presence in at least 100 of the 191 communes and therefore had substantive impact on an estimated 50 000 households, comprising 285 000 people.
Under rural infrastructure, the roads and bridges sub-component set out to facilitate transport of people, farm inputs and produce and the outreach of social services. 105 km of commune-village roads and 7 suspension bridges were built as planned, and only the training activity fell short of 100% achievement, although the database system implemented by TA is not now operational. Physical results are impressive, but: designs were deficient; villagers engaged for construction were reported to be paid lower than market wages; community dedication to o&m is doubtful; and district budgets for works are unlikely to be sufficient to cope with costs to maintain accessibility for increasing traffic.
For domestic water supply the enormous problems of provision in the karstic limestone areas of Zone I -- and the potential solutions -- were correctly identified, but the optimal designs were not followed. Thus, although the sub-component achieved 100% of planned physical activity, covering over 5 000 new rain water tanks and 728 rainwater jars, repair of 56 communal tanks and construction of 12 gravity flow systems in 24 Zone I communes, there were serious weaknesses in design, construction quality, capacity for storage and water quality assurance; water user groups do not collect fees for repairs; a water supply database set-up by TA is not operational; and technical capacity, staffing and means of transport at district level are still insufficient.
The provincial irrigation scheme improvement work was largely pre-targeted rather than participatory and claimed farming benefits may be exaggerated. However, in all, 39 provincial schemes covering some 2 840 ha and 134 micro-schemes, including 20 new, covering 2 030 ha were assisted, with an additional water supply to over 4 300 ha and thus 100% or higher achievement of actual physical works, equipment supply and training -- providing potential benefit to over 7 300 households in 90 communes. For micro-schemes, feasibility studies were non-existent or inadequate; cost benefit analyses were not done; dams were constructed at high cost; siting and design were in many cases questionable; and little attention was paid to quality of construction and supervision and water user group establishment. On visits, several concrete canals were silted or blocked by landslides and dams were not de-silted, clearly indicating neglect of proper operation and maintenance.
Under the agricultural development component, farmer extension objectives were consistent with small farmer needs and successful insofar as yields and outputs of major cereal crops were markedly increased and interventions in livestock, forage production and agro-forestry demonstrated -- or show promise of -- expanding uptake. However, design of models appears to have ignored the value of indigenous practice, the constraints of the poorer, mountainous areas and the ability of households to cope with high cost systems. As a result, numbers of farmer field school (FFS) and other formal groups fell far below target - at 25% for FFSs - but were compensated by establishment of a wide village based extension network and on or near target performance of most of the equipment supply and training inputs. In total, some 370 FFSs and IPM clubs were formed, covering over 3 200 households in direct membership, with 266 demonstrations of improved technologies and over 80 000 person days of farmer training. Although training of staff and farmers has been impressive in numbers, the evidence now is that few FFSs or groups have survived; that there is little conviction about the methods used; and that the facilities provided and impetus achieved have not been maintained.
The animal health sub-component was implemented with: beneficiary involvement in selection of commune animal health workers (CAHWs); commune-based operation of services; provision of a necessary minimum of equipment and drugs for CAHWs; and their practical training. In total, 474 CAHWs were trained and deployed in all 191 communes, basic furniture, equipment and drug supplies provided, 9 district and 2 border veterinary stations established and a revolving vaccination fund for notifiable diseases supported, registering virtually 100% achievement. Weaknesses of the component have been at district and provincial level where there is limited evidence of maintenance or continuity of service from the equipment and training supplied. Gains in livestock outputs are difficult to gauge, but availability of vaccines and antibiotics will have had significant impact on household incomes.
The forest protection intervention recognised the importance of watershed protection and soil and water conservation. Beneficiary households gained income in return for vigilance in protection. Operation of protection contracts and achievement of objectives has been fair and at a cost of VND 8.7 billion (USD 0.55m), the economics are reasonable. 20 000 hectares of critical watershed have been protected, with over 4 100 households in 109 villages receiving direct benefits; and enrichment and new planting have reached 973 ha, or several hundred percent of target. All other aspects achieved an out-turn of 100%. The major shortcoming has been the failure to address the serious problem of exploitative arable use of steep lands and consequent degradation in many upland catchments.
The income diversification component was confined to women and seen as crucial to poverty alleviation. The implementing agency was the Womens Union (WU). Despite the claimed 100% achievement of equipment supply and training inputs, performance was well below expectation. Opportunities were missed to create linkages between saving and credit, labour saving technology and education. PRAs were undertaken in 60 communes and 318 womens savings and credit groups formed. Of these, 118 groups covering 1 538 households are still active but have little substance as groups and saving activities are absent or negligible. Total borrowing to date is VND 4 606 million (USD 287 000) or 67% of target and repayment performance so far is reported to be satisfactory at 86%. However, the training of local people has not been effective; few diversified businesses have been set up, and many have been unprofitable; and the channelling of IFAD credit through the quasi-commercial bank VBARD was not a viable approach.
In the social development component, education was the largest activity, addressing the problems of adult illiteracy and primary enrolment, especially of girls. Education continues to be extremely important as the key influence on awareness and understanding, which are the fundamental pillars of ethnic group advancement. Major achievements have included: education development plans for 70 communes; construction of 146 rooms for commune schools and 7 district continuing education centres; supply of equipment and material; and bilingual and refresher training for over 3 200 teachers. School construction has been satisfactory, but quality of workmanship was poor in some cases and proper maintenance and upkeep are absent. Most equipment and materials were of good quality and useful. There is evidence already of improvement in school access, enrolment, attendance, retention and narrowing of the gender divide. Key shortcomings have been: limitations of participation; improper use of PRA techniques; lack of upgrading to make libraries functional; and failure of the trained women literacy volunteers to activate village literacy and other programmes as envisaged.
Objectives of the health sub-component were germaine to the predicament of poor. Major achievements were: networking and training of 801 village health workers (VHWs); upgrading of facilities and equipment in 4 district health centres; construction and equipment for 5 commune health centres; training and refresher courses for doctors and other staff; and free supply of drugs. Most aspects reached 100% of target. VHWs have transformed outreach and prevention in remote villages; already reduced incidence of malnutrition and diseases is recorded; and significant increases are noted in in-patient and out-patient numbers at hospitals. Remaining concerns are: sustainability of drug supply; failure of drug revolving funds; the low percentage of women VHWs; lack of linkages between traditional birth attendants and VHWs; low educational standards and skills; and low allowances for VHWs, are issues that require attention. VHWs need significant further refresher training and capacity building and simplification of their reporting formats.
Introduction of the commune development fund in 2001 enhanced community participation, allowing CDBs to become more effective and ensuring at least some gender-balanced village representation. CDFs have spread from 10 to 78 communes, all with poverty rates of over 30%. Emphasis of CDF activities has been on infrastructure: 128 inter-village roads, 7 suspension bridges and 42 culverting or small improvement schemes; 56 water supply facilities and 289 domestic tanks; 184 irrigation construction or rehabilitation projects; one hydro-electricity dam and power cabling; and provision of 55 class rooms, 142 branch schools, 2 boarding hostels and associated furniture, have been completed. Other main interests have been in income diversification, supply of farming inputs, particularly improved seeds, forestry and herbal plants and technology transfer models.
Project management has been proficient as evidenced by the virtually 100% achievement of physical and financial targets and general progress. The provincial institutional arrangements functioned reasonably well and district structures and arrangements set up by and supported by HPM have proven to be effective. Key remaining weaknesses have been that: DPCU chairpersons have changed frequently and not devoted adequate time; CDBs and VCUs are yet to become properly established; coordination problems continue to exist, notably at districts; and appraisal, supervision and convergence between components and with other projects, as well as monitoring, have been inadequate. UNDP and SIDA TA has been important; premature withdrawal of the SIDA inputs, for reasons of unsatisfactory performance and management disagreement; and financial cuts for UNDP inputs due to budgetary restrictions, have had negative impact on the Project. The monitoring and evaluation system has captured financial and physical progress, but has had limited application as a management tool, in supervision, or as an indicator of impact or gender or ethnicity balance. Use of information technology is limited and simplified standardized formats would significantly improve the quality of monitoring and reduce the tedium of the work involved.
IFAD specific concerns. In the context of poverty targeting, since 90% of the Ha Giang population is ethnic, targeting of ethnic minorities is not a meaningful concept; so HPM concentrated on targeting on the basis of poverty level. Macro-targeting of activities is assessed as adequate, but not all components were designed to target the worst off, for instance provincial irrigation scheme improvement and use of VBARD for credit inevitably entailed disproportionate benefits for better-off groups. Targeting of communes, villages and sub-projects was generally satisfactory but if literacy and labour availability were pre-conditions for participation, the poorest, in particular women headed households, could be under-represented. Although all HPM components focused on priority needs, impacts on target groups would almost certainly have been more significant if activities had been targeted to fewer communes; and carried out in a more integrated, component-linked manner.
As far as gender is concerned, HPM has had reasonable success in addressing women's practical needs for access to services and markets, absence of gender dis-aggregation in PRAs prevented serious constraints for women such as violence due to alcoholism of husbands, excessive workloads and reproductive health problems being identified and addressed. Women's strategic needs, such as managerial and entrepreneurial capabilities and involvement in decision making remain unfulfilled, despite WU efforts and their mandatory incorporation in village and CDB affairs.
There was little scope for proactive participation or community involvement by the poor in identifying needs and shaping Project design. Because PRAs were mostly carried out by each component separately and often were simple line department surveys to identify farmer requirements from that department and activities selected in a prescriptive way, they were little used for targeting or monitoring. Improved use of PRAs in the context of CDF sub-project selection and planning showed their value, which is now recognised by province and project management as a means of strengthening decentralisation and democratisation processes. Although the province has been supporting decentralisation, it appears that participation of the local population, and particularly women, in decision-making has been limited. Local social self-help mechanisms and informal organisations have not been capitalised on; devolution of financial control has yet to materialise; and existing commune institutions could have been used better to sharpen gender and poverty focus and enhance the sustainability of Project activities and impacts.
Rural Poverty Impact
Overview. As far as practicable, HPM focused on the poorest communes; the target group poor - and women - are believed to have accounted for between 35 and 70% of the participants for most activities. For income diversification and credit women were prime beneficiaries, but they were under-represented in the CAHW and VHW cadres. Direct beneficiary numbers derive from M&E records and Project reports and indicate that between 3 000 and 17 000 households participated in the main individual Project activities, giving a gross total of benefit incidence of 136 000 households. Accounting for overlap and duplication, it is estimated that at least 50 000 households have received substantive benefits from one or more HPM interventions.
Province wide, incidence of poverty, is recorded as having fallen from a head count of 34% in 1997 to 16% at the end of 2003, using a poverty line equating to a household income of some VND 5.5 million or USD 345 per year. HPM has been one of the main factors in this improvement, alongside a raft of poverty-oriented policies and provincial programmes. Economic statistics put provincial GDP per head in 2002 at VND 2.14 million, equivalent then to about USD 165, compared to VND 1.07 million - USD 91 - in 1997, an 80% increase in real terms; agriculture, forestry and fisheries still made up 48% of GDP in 2002. In confirmation, the recent Ha Giang Participatory Poverty Assessment, reported significant improvements in the predicament of the poor in the last five years, but that village poverty head counts were higher than estimated by commune authorities and that considerable further scope for poverty alleviation existed.
Impact on Physical and Financial Assets. The impact has been positive and substantial -- from agricultural productivity and surplus produce for sale; from forestry protection revenue and rights of use; to a lesser extent from income diversification; from the facilitation of market access and potable water provision. The quantum of benefits and wider farm and household impact have been estimated for two simple farm and household models:
for a small farm with 0.5 hectares of single season paddy, the effect of Project technology adoption would lift the annual yield from 2.5 t/ha to 3.8 t/ha, that is from 1.25 t to 1.90 t, or from bare basic food subsistence level to a surplus of 0.6 t/year, worth some VND 1.2 million (USD 76) at current prices and possibly adding between 25 and 50% to family cash income; and
for a similar farm where irrigation improvement allows a second crop of paddy to replace a rain-fed spring crop of maize on 0.5 hectares, the effect would be an output of 1.25 t of paddy worth VND 2.5 million to replace an output of maize of 0.7 t worth VND 1.6 million, that is a benefit of VND 900 000 gross - and probably VND 600 000 (USD 40) net of additional direct costs - in just one season.
Similar or greater incremental benefits than these would have been attained for the majority of the Project interventions in crop and livestock enterprises; and those from forest protection, with only minimal exploitation of firewood and non-timber forest products, are estimated to equate to VND 486000 (USD 31) per household per year. These gains are manifested in: rights of use of additional and/or better irrigated land; possession or assured use of tree plantations; ownership of livestock; better housing; reduced indebtedness; ability to pay school boarding fees and buy shoes and clothes; and ownership of furniture, utensils, farm tools, bicycles and radios. Some 30% of first hand beneficiaries under the productive components would be enjoying more than one of these tangible livelihood gains. Project contribution to increases physical and financial assets is substantial, 3, on a scale of 1 to 4.
Impact on human assets. The Project has had some positive effects on human assets in terms of health, nutrition, education and work burden alleviation, as well as building confidence and self-reliance, but overall it is a mixed picture of achievement. The agricultural and productive interventions have contributed to adequacy of food provision and availability of cash income, primary determinants of nutrition and accessibility of health and education services. Literacy initiatives have had much less effect and the impact of drinking water supply has been chequered. Probably the major change for the better in human assets has been the outcome of providing the necessary information and support to raise the level of awareness, knowledge and skills, although there is still a long way to go to make this fully effective. Evidence of change in human assets is seen in: the proficient uptake, practice and dissemination of activities; the level of participation and interest in new developments; the success of the CAHW and VHW systems; the ability of men and women farmers to present themselves and their views, increasingly in a social and political as well as a technical context. The contribution of HPM to the increase in human assets in terms of direct health benefits is substantial, 3; in education, modest to substantial, 2-3; and that of water supply effects is modest, 2; for the impact on women's workload, modest, 2; for the imparting of skills, knowledge and self-confidence, as described, and the indirect effects on improved human assets from increased food sufficiency and incomes, the contribution is substantial to modest, 3-2.
Impact on social capital and empowerment. The Project has had only a limited impact on social capital formation and people's empowerment; there remains a large gap between what is possible and desirable from a development viewpoint in terms of participation and community self-reliance and involvement and what has been attained up till now. The impact that has been achieved has emanated partly from the demand driven and group approach adopted in the later and peak years of implementation; and has resulted partly from the uplift of financial and human assets noted above. Strong efforts have been made in farmer, womens and users organisations and in attempts to ensure gender balance and to address womens needs, but these have met with only marginal success and their impact is therefore ranked as modest, 2.
Impact on food security. Application of Project techniques has lifted crop yields and outputs significantly and typically moved poor, small farm beneficiary families from a position of seasonal food insecurity, to one of food self-sufficiency in most years. Estimated yield increases are of the order of over 40% -- and up to 100% in better areas -- for paddy; and 10 to 30% for maize and soybean. The farm and household models indicate that these yield increments would have ensured an output above the immediate family needs for basic energy foods and the ability to spend and invest to improve future farming operations and therefore help to break the vicious circle of poverty. Impact on food security is categorised as substantial, 3.
Other impacts. The impact on the environment where the Project has been active has been positive. However, in terms of the overall need for cessation of the profligate and exploitative arable use of steeply sloping lands and for upper catchment protection, the present coverage can only be regarded as a holding exercise. The Project has had limited impact in community-based natural resource management, or in wider application of soil conservation measures and agro-forestry. On the negative side, the intensive crop husbandry methods and the heavy fertiliser and agrochemical use that have been advocated pose dangers for both long term soil fertility and human intake of the foods produced; and the inevitable blasting and earth moving for roads construction may contribute to landslide and soil erosion propensity. Environmental impact is classified, for the present, as modest, but potentially negative in the aspects mentioned; however, the Project contribution to a changing and more stable situation and to an enhanced awareness of the dangers, is assessed as substantial, 3.
The impact on institutions, policies and regulatory framework has come from working through the organs of district governance, as well as with the local private sector. The Project has instilled a degree of orientation, training and experience, as seen in increased effectiveness of extension and the start of use of the PRA as a basis for local planning. The provincial authorities expressed their satisfaction with the level of success of HPM, so much so that the approach and design had already shaped the future development strategy and policies of the province; however, impact on district and commune institutions has been less marked. Respondents among Hanoi agencies accredited IFAD with having influenced the development philosophies, principles and concepts of other donors - and government itself - in the adoption of pro-poor policies and instruments in project design but were critical of the lack of a permanent or consistent IFAD presence in the country and the limitations of the IFAD system of supervision and liaison. HPM has thus played some part in the evolution of new policies and regulations and has utilised approaches and techniques that could influence policy and regulation formulation in future. Its current impact in this regard is classified as modest, 2.
Innovation, sustainability and overall impact. HPM had elements of successful innovation in its agricultural and environmental interventions, particularly with regard to extension methods, its community based approach and use of the decentralised local governance system. It was less so for the infrastructure activities, where accepted top-down practices that were already of dubious effect and sustainability were followed. The rating for innovation and potential scope for replication of Project approaches and activities is substantial, 3. Despite its technical proficiency, HPM has been less successful in inculcating conviction, responsibility and capability among line departments and local user groups and beneficiaries. These are key elements of future sustainability, which is also a function of the external conditions, including those of climatic change and national and international market conditions; prudence dictates that predictions of sustainability should be moderated to account for their influence. Accordingly, sustainability of Project activities and benefits is assessed as equally balanced between modest and substantial, 2/3. The overall impact of the Project is similarly ranked as between modest and substantial, 2-3.
Performance of the project
Relevance of goal and objectives. The goal of the Project, to improve the living standards of the target group, which is mainly composed of ethnic minorities living in marginal upland areas is a fair interpretation of development needs of the outlying and mountainous districts, where, in the early 1990s, many tribal groups were living in a state of serious deprivation and precarious livelihoods, as some still do. The relevance of the goal was - and is - high, 4. The objective of the roads and bridges sub-component was to improve pedestrian access in remote upland areas, the implied outputs including increased skills and sensitisation of local infrastructure agencies and improvement of village access for trading and social purposes. The relevance of these objectives is rated as high, 4. For domestic water supply, the objective was to improve both access to and quality of drinking water where availability was problematic; the relevance was therefore high, 4. For irrigation, there was a two-fold objective: to increase the area of irrigated paddy; and to improve irrigation management. The former aim has been realised to a reasonable degree; the latter, not yet so - and the relevance, for the larger, provisional schemes is regarded as modest, 2, but for micro-schemes, high, 4.
The specific objectives of agricultural development component were to develop and implement effective extension, leading to increased agricultural, livestock and forestry production; to improve delivery of animal health services; and to better manage critical watershed forests. These objectives were apposite to the predicament of forest communities at appraisal and continue to be germaine. Relevance of these objectives is high, 4. The objective of the income diversification component was to increase and diversify household income by identifying and credit funding a range of micro-enterprise options. The relevance was high, 4; the purpose is still appropriate, but the scope for, and means of achieving income diversification now need to be seriously re-assessed. The objectives for social development were to improve access to, and quality of primary education, adult literacy and vocational training programmes; and to improve health by strengthening services. The implied aims and expected outputs of the education and health activities are all crucial to the Ha Giang situation and relevance is assessed as high, 4. By the same token, the objectives of Project management and coordination, although stated at appraisal in a vague and narrow way -- and those attributed to the CDF -- basically of widening and democratising sub-project selection and design for commune and village level interventions, were -- and are -- strongly relevant, 4.
Effectiveness. The goal achievement for the whole Project can be estimated from progress in arresting natural resource degradation, lifting the order of farming productivity, food security and income; and measurably improving living standards by provision of infrastructure and social services. Despite variation in component impact and much scope for further improvement, aggregate effectiveness is assessed as substantial, 3. The roads and bridges sub-component has accomplished its target for works and expanded its impact through the CDF. Effectiveness of meeting the roads objective is modest to substantial, 2-3. Domestic water supply has been subject to great difficulties; designs were wrong, construction standards poor and o&m, including ongoing funding, neglected. Only limited improvement of access and reduction of work load for collection were achieved; and training and mobilisation of WUGs inadequate. Effectiveness is assessed as modest, 2. In the irrigation sub-component, the provincial schemes improved and expanded were pre-selected and misgivings persist about the actual scale of improved crop production, but effectiveness is reckoned to be substantial, 3. For micro-irrigation, absence of proper feasibility and cost/benefit studies, inadequacy of siting, over-emphasis on high-cost concrete structures and uncertainty about crop production benefits render the effectiveness as minimal, 1.
The agricultural development component has largely achieved its objectives, transforming the predicament of adopting farmers and livestock keepers and providing a clear example of application of available technologies through creative extension. For forest protection specifically, effectiveness has come at high cost and more use of forest products could have been made. However, considering the original scenario of low crop and livestock outputs and the extended hunger gap for many families, the aggregate effectiveness is considered high, 4. This conclusion is corroborated by provincial statistics which show crop area, yield and livestock number increases of from 11 to 38% since 1997. Income diversification is seen to have had limited impact, with a non-viable credit scheme, chequered experience of labour saving technology introduction, lack of linkage and failure to achieve meaningful diversification of earning or substance in group formation; effectiveness is rated as negligible, 1. In the case of the social development component, the education sub-component has fulfilled its physical targets in facility, equipment and supplies provision with noticeable impact of training and software inputs in enrolment and retention, including those for girls, a major advance. The health sub-component has achieved similar and comparable results, particularly in VHW training and mobilisation, with patient numbers increasing, adequacy of district services enhanced and child malnutrition and disease incidence falling: recent provincial statistics show increases of 14% in primary girls enrolment and 45% in patients treated and a fall in child malnutrition of 40% since 1997. The weaknesses in both education and health - of local agency capability - are being ameliorated by participatory activities under the CDF, so that in terms of the parameters of participation, community involvement and gender balance, effectiveness is improved. Accordingly, for all three of these sub-components, effectiveness is assessed as substantial, 3. Operations under the Project management and coordination have worked reasonably well but impact is still only tentative in some applications, as in the case of M&E, support of PRAs and gender and district and commune capability. Effectiveness is assessed as high, 4 at provincial level - but only reasonable, substantial, 3 for downstream activities.
Efficiency. The Project in total has entailed deployment of substantial resources and funds, particularly at start-up. The crux of efficiency is the quantum and likely continuity of benefits from the high level of costs incurred. Re-calculation of the internal rate of return, based on highly conservative estimates of uptake and benefit, show an EIRR of 15%, compared to EIRRs of 14% to 27% for key components at appraisal; thus the efficiency of the whole Project is put at substantial, 3. For rural infrastructure, cost disparities and quality of design, construction and o&m for individual intervention in roads, bridges and water supply; and excessive costs for irrigation schemes, dictate that the rankings for efficiency be put at substantial, 3, for roads and bridges; modest, 2, for water supply; and minimal, 1, for irrigation. For agriculture, present and prospective expenditure is considerably below budget, partly due to cost containment strategies, and efficiency is substantial, 3. In income diversification the low costs are due to low uptakes of credit and enterprise development; given the doubtful profitability of investments, the emphasis on traditional pursuits and the unlikely sustainability of activities or benefits, efficiency is ranked as negligible, 1. Under social development, the education and health sub-components both achieved target performances well within the allocated funding in real terms. Education has registered meaningful gains in impact in most activities and efficiency is graded as substantial, 3. The health sub-component has been characterised by timely and effective procurement, supply and sound training and deployment of VHWs and other medical staff. Despite the questionable sustainability of the free drugs policy, efficiency is assessed as substantial, 3. The Project management and coordination component has incurred excessive costs and still has some weaknesses, but has had significant success in operating a complex and challenging project, to the extent that it has strongly influenced the design and conduct of similar development interventions by government and other donors. Efficiency is put as substantial, 3, a finding confirmed by the overall Project performance as assessed by UNOPS.
Performance of partners
The IFAD process of formulation and design was measured and reasonable but the concept and content should have been less ambitious and complex. During implementation, the experience of the PSC and PPCU and government -- despite criticism from SIDA and UNDP -- has been that back-up has been timely and effective; performance is ranked between modest and substantial, 2/3. As cooperating institution, UNOPS, for the first four years was considered proficient but in the last two years, supervision has declined in rigour of oversight and progress monitoring; performance is assessed as only adequate, 2/3. Central Government provided dedication and continuity in Project oversight, policy guidance and donor and ministerial liaison. Provincial bodies have played a crucial role in enablement and support of project activities. Major district weaknesses have been stringency of finance and resource availability; at commune and village level there have been rather more problems of competence of staff and lack of resources; overall, government performance is satisfactory and substantial, 3. Project management, comprising the PPCU, and DPCUs, has shouldered major responsibility for driving forward local interest and participation with dedication and competence. Routine tasks of planning, budgeting and progress monitoring have been reasonably performed, with only limited weaknesses in M&E and component management. Downstream agencies became increasingly involved and confident. In aggregate, Project management performance is assessed as substantial, 3. There is a very limited presence of NGOs and the Project has therefore made little use of their expertise, but has had good liaison where existing NGO activities overlapped with HPM interventions. The Project itself has been a major player in development of CBOs and groups of various kinds, but their sustainability has been marginal. The NGO/CBO contribution to support of HPM is ranked as modest, 2. Pre-existence of the SIDA funded MRDP was a useful precedents for design of HPM and the involvement of SIDA in the planning, design and initial stages of implementation was useful and positive. However, failure to resolve practical problems of integration of the two projects and deficient communication on funding and operations led to collapse of arrangements and withdrawal of some TA inputs; performance of SIDA as a co-financier and provider of TA is rated as modest, 2. The approach and support of UNDP has been effective in both funding and technical assistance and its performance as a co-financier is put as substantial,3.
The impact and performances of the Project and partners are summarised in the tables below.
| Whole |
| Project - All |
|Component/ Activity|| |
| Rural Infrastructure: |
- Roads and Bridges
|- Water Supply|| |
|- Irrigation - provincial|| |
|- micro|| |
|Agricultural Development|| |
|Income Diversification|| |
| Social Development: |
|- Health|| |
|Commune Devpt Funds|| |
|Project Mangmt/Coordn|| |
|Whole Project|| |
Individual recommendations of key significance for design and conduct of future projects are:
HPM has been a success in most respects, but was complex and over-ambitious in content;
future projects should be more closely tied in to provincial and district programmes;
further development should be restricted to the poorer, more remote communes;
pre-determined activities should be limited to fundamental poverty alleviation areas of family food self-sufficiency and adequacy of cash income generation from established and traditional household enterprises;
livelihood financing assistance for poor households should be based on matching grant, seed capital or simple, deferred repayment, rather than credit, arrangements;
the CDF approach should be used as the main plank for determination of project activity across the technical and social sectors for any follow-on intervention, but with strong provisos as to the determination of feasibility, better technical oversight and real evidence of community ability and commitment for proper operation and maintenance;
capability building, training - and the gaining of practical experience through project activity - in participation for both community and formal local organisations should be major concerns;
assistance to government would be greatly enhanced by direct IFAD supervision of its Vietnamese projects; or by contracting supervision, with the agreement of government, to a national, or joint national/international, professional consultancy or development agency;
if the successes of HPM can be built upon without further dissipation of initiatives and impact through delays in project preparation and approval - and the foregoing recommendations are properly incorporated in the design and in subsequent project operations - a follow-on project would be feasible and justified; and
IFAD is to engage directly and indirectly in policy dialogue at the provincial and central level, playing a catalytic role though the Project and in close consultation and co-ordination with other international organizations. By building on evidence from the ground, IFAD could further increase and deepen its contribution to the evolution of policy in Viet Nam, especially with respect to decentralized approaches, stronger local participation and community empowerment.
1/ The Interim Evaluation Mission comprised: Mr Somesh Kumar, Education, Health, M&E and Management Specialist; Mr Michel van der Stricht, Rural Infrastructure Consultant; Mr Sarath Mananwatte, Economist; Ms Sarah Mader, IFAD Associate Evaluation Officer for Community and Women's Involvement and Mr Jim Semple, Team Leader and Agriculture/Institutions Specialist. The Mission was supplemented by the national counterpart team comprising selected members of the Provincial Project Coordination Unit and District cadres. Mr Ashwani Muthoo, the IFAD Senior Evaluation Officer, joined the Mission in the last week of its work in Vietnam. The National Wrap-up Meeting was chaired by the Vice Director General of the Department of Agricultural Economy, Ministry of Planning and Investment and also attended by Mr Mattia Prayer Galletti, the IFAD Country Portfolio Manager (CPM), Ms Atsuko Toda, the designate IFAD CPM and members of the formulation mission for the proposed follow-on project as observers, as well as 20 key decision makers from the two provinces and from the concerned ministries and partner agencies.
Agricultural Resources Conservation and Development Project in Quang Binh Province (2004)
Interim Evaluation 1
The Agricultural Resources Conservation and Development Project (ARCDP) was financed by an IFAD loan of USD 12.8 million with an additional UNDP/TA grant of USD 1.4 million and a borrower contribution of USD 1.9 million. The loan became effective on 15 April 1997 and the closing date was extended by six months to 30 December 2002. The IFAD Interim Evaluation is being conducted as a mandatory exercise prior to the consideration of a second phase of the project. The evaluation follows the methodological framework set out by IFAD's Office of Evaluation. Primary data was collected through: (i) a series of participatory focus-group discussions with semi-structured questionnaires; (ii) case-studies of individual households and grassroots institutions based on interviews; (iii) open-ended interviews with key informants at all levels. Preliminary conclusions were presented at wrap-up meetings in Quang Binh and Hanoi and are due to be finalised at a workshop in Hanoi in early June.
Two important conditions should be noted: (i) The project had been closed for around fourteen months at the time of the Interim Evaluation Mission. Thus the technical implementation units and some of the grassroots organizations set up under the project were no longer functioning, and the project management unit consisted only of a residual presence now funded by the province. (ii) No independent impact assessment or household survey had been commissioned by the project. While the official commune-level statistics available are remarkably full, the trends that can be observed therein result from a multiplicity of interventions of which ARCDP is only one.
Main design features and implementation results
Project rationale and strategy. In line with IFAD's overall strategy in Vietnam, the ARCDP was founded on an area-based, multi-sectoral and single-province approach. The rationale was that the reduction of poverty in Quang Binh might best be achieved by a mix of sectoral interventions implemented through the government departments, with community priorities identified through participatory processes involving all potential beneficiaries. The design of the project sought to develop participatory processes, encourage community ownership of schemes through users groups and self-management boards, assist in the creation of effective extension services, and leave behind it government departments with enhanced capacities and awareness.
Project area and target group. Some or all project activities were targeted at all communes in the seven districts of the province. Certain components, such as road rehabilitation, irrigation schemes and agricultural extension, were designed to benefit all members of the communities concerned, while others (sand dune fixation, credit) were to be targeted specifically at the poorer families. The total number of beneficiary households was calculated at around 65,000. The approach to targeting in project design was markedly flexible, being conceived as ‘a continuing process according to the implementation plan', a rather imprecise indication. In terms of districts and zones, the two northerly districts of Tuyen Hoa and Minh Hoa are the districts with the highest rates of poverty. Very poor communities are also to be found in the sandy areas of the coastal belt. The bulk of project interventions were targeted to these two regions.
Goals, objectives, components. The goal of the ARCDP was the sustainable reduction of poverty, and the primary objectives were the increase of household incomes, improved food security and wellbeing, and reduction of isolation. Secondary objectives were a reduction in the provincial food deficit, the protection of property from shifting sand dunes and an enhancement of the capacity of technical services. The seven project components were: irrigation rehabilitation, agricultural development, aquacultural development, sand dune fixation, road rehabilitation, rural credit and institutional support. Specific objectives by component included: the expansion of the irrigated area by around 600 ha, increased agricultural and livestock production and productivity, increase in marine and freshwater products, the fixation of 4,000 ha of sand dunes as well as the provision of cash income for the communities involved in afforestation, the linking of isolated communities to each other and to markets through the rehabilitation of rural roads, and the improvement of the reach and capacity of extension services.
Implementation oartners and arrangements. Project implementation was to be carried out by the Water Resources Department (WRD), the Department of Agriculture and Rural Development (DARD), the Department of Fisheries and Aquaculture, the Department of Transportation (DOT) and the Department for Science, Technology and Environment (DOSTE), and coordinated by a Project Coordination Unit (PCU) set up within the Department of Planning and Investment (DPI), whose parent ministry (MPI) acted as the Executing Agency for the project. Disbursement of project funds at provincial level was managed by the Department of Finance (DOF). A Sub-Loan Agreement was signed with the Vietnam Bank for Agricultural Development (VBARD) concerning the implementation of credit activities. Technical Implementing Units (TIUs) were to be set up within each implementing agency at provincial and district level to coordinate, supervise and monitor project activities. UNOPS was to act as Cooperating Institution in charge of loan administration and project supervision.
Changes in policy and institutions. Government policy concerning rural development has been marked by a growing commitment to poverty eradication. The Comprehensive Poverty Reduction and Growth Strategy (CPRGS), a 10-year action plan adopted in 2002, outlined a growth-based strategy for poverty reduction, with policies covering macroeconomic, structural and sectoral areas, and an identification of priority programmes designed to feed into the annual investment plan. Proposals include increased resources to improve the research and extension system, enhancing access to credit for the poor, improving security of land tenure and sustainable management of natural resources with the involvement of all stakeholders. In terms of government institutions, there has been one significant change, the creation in July 2003 of the Department of Natural Resources and Environment (DONRE) which assumed certain functions previously carried out by the Department of Science, Technology and Environment (DOSTE).
Design changes during implementation. The major design change was the creation of the Community Development Fund (CDF) component, approved in June 2001. Two-thirds of the CDF budget was drawn from funds of the under-performing credit sub-component and the remainder from savings in equipment procurement, training and project operation, together with a previously unallocated sum. The stated purpose of the CDF was to sharpen the poverty focus of the project and more directly to meet the priorities of communities expressed through PRA. The closing date of the project was shifted from 30 June 2002 to 31 December 2002. The proposed involvement of the World Food Programme in financing food-for-work operations, to a total of USD 2.5 million, did not materialise.
Main implementation results. The overall financial target of the project was met, with a full disbursement of the IFAD loan amount and the UNDP grant. The only substantial under-achievement in financial terms was under the credit component. In physical terms, the project achieved its main objectives by: (i) expanding the irrigated area by 1,306 ha and upgrading irrigation facilities for a further 2,236 ha; (ii) establishing and strengthening breeding stations and AI centres and improving the capacities of staff and beneficiaries; (iii) establishing a commune-level agricultural extension system and a network of demonstration sites; (iv) investing in shrimp hatcheries and fingerling stations together with appropriate demonstrations; (v) organising the afforestation of 5,000 ha of sand dunes by 2,600 selected households, who were remunerated in cash; (vi) building or upgrading 166.3 km of good quality rural roads and constructing a ferry and two large bridges; (vii) establishing a project management and coordination structure as designed; (viii) producing the required annual work plans AWP&Bs based on the results of PLA/ PRA; (ix) putting in place an M&E system and carrying out participatory M&E exercises; (x) following the guidelines and regulations concerning the procurement of goods, fund withdrawals and reporting systems and the submission of financial and M&E reports. In terms of training and capacity building, a full programme of courses was implemented under each component. Users groups and self-management boards were set up for the various activities to ensure the participation of communities in the implementation and maintenance of the various schemes.
Problems in implementation. The main areas for concern were as follows. (i) The credit component was largely ineffective from the start, mainly due to weaknesses in design, which failed to take into account the availability of credit from other sources at lower interest rates, the need for suitable incentives for the lending institution (VBARD) and the lack of legal status of the savings and credit groups. Only 28% of the budget was disbursed and only around 4,000 households received loans. (ii) The average actual per km cost of the upgraded roads was about 80 percent higher than the planned per km cost. (iii) Under the aquaculture component, only 60 ha of shrimp ponds were upgraded as against the planned 400 ha, only 60 of 5,000 fish cages were constructed and an effective extension system was not put in place. (iv) Because of the weak performance of the credit component, the proposed expansion of income generating opportunities did not take place, affecting in particular the planned increases in livestock production.
Rural poverty impact
Overall impact assessment. Official statistics show a reduction in the proportion of poor households in the province from 36% to 21% in the six years of project operation. Yield increases of around 5% are indicated for the land covered by upgraded irrigation schemes; the benefits of rehabilitated markets and roads are reflected in unquantifiable increases in economic activity. Afforestation in the dunes provided direct income to poor households, as did the various employment opportunities under the project. Provincial figures show an 11% increase in numbers of pigs and nearly 50% in numbers of poultry (the two kinds of livestock favoured by poorer households) between 1996 and 2002. Access to schools, health centres and markets improved as a result of roadbuilding, and the communities located close to the rehabilitated markets benefited from better provisioned and more frequent markets. On the other hand, the relative failure of the credit component entailed a dearth of the new income-generating opportunities for the poor which were envisaged in project design.
Impact on physical assets. Irrigation schemes resulted in increased agricultural production (mainly rice) for an estimated 12,000 households. In many cases, the yield increase was much greater than the reported average of 5%. The remuneration of local labour entailed considerable cash earnings, notably in the construction of irrigation works and for tree planting and maintenance in the dune areas, where the selected households were able to earn up to VND 20 million in two years. A total of VND 7 billion was paid to labourers for work on the new roads, although local labour was not always preferred by the contractors, despite requirements set out at project design. Households interviewed by the mission in six districts (including the poorest districts of Minh Hoa and Tuyen Hoa) reported an average reduction of the period of food deficiency of around three months.
Communications. The new roads have contributed significantly to socio-economic development in isolated communes in mountainous and coastal regions, directly affecting perhaps 55,000 households. The movement of vehicles has increased substantially, with estimated increases of 3,000 percent, 600 percent and 300 percent for bicycles, motor cycles and pick-ups respectively. Improved access to markets has caused reductions of up to 100% in the price of agriculture inputs as well as substantial increases in prices paid to the farmers for their produce. In some cases, new settlements have grown up along ARCDP roads.
Training. Training and capacity building constituted an important aspect of the project, and nearly 400 training courses were conducted by the project on animal husbandry, cultivation, integrated pest management (IPM), irrigation, road management and afforestation. Courses in IPM and in the raising and plantation of seedlings under the sand dune fixation component were particularly appreciated.
Gender impact. The mission found with the exception of the issue of land titles for women little evidence of discrimination against women in project activities. Women participated fully in training courses, PRA exercises and participatory M&E. The proportion of women in SMBs, WUGs and project management boards were estimated by the project at 25%, 37% and 21.5 % respectively, while 70% of all workers in the sand dune fixation activities were women. Outstanding gender issues relate to inequality within households and to problems of empowerment rather than income: while women often manage the money, the traditional dominance of husbands in decision-making is unimpaired. As mentioned above, the practice of granting of land use certificates in the husband's name prevents women from accessing formal credit, and the news of legislative changes, with titles issued jointly to husband and wife, has not filtered through to the villages. It is felt that it is the effects of traditional culture and prejudice that underlie gender discrimination in Vietnam, rather than bias in government policy.
Water supply, health, education. The provision of reliable and uncontaminated water supplies under CDF has had a positive effect on the incidence of diseases of the eyes and skin, as well as on dysentery and typhoid. Primary schools and kindergartens were also constructed under CDF. Schools reported a reduction in drop-out rates, an even gender balance and better health.
Participatory development. Training in PRA was given to nearly 300 workers at district and commune level and PRA exercises were carried out for all components. However, there were weaknesses in terms of the purpose and outcome of these exercises, with PRA often conducted by line departments in order to seek approval and support for predetermined activities. Intensive social mobilisation was carried out by a handpicked team in three pilot villages, where significant advances were observed in terms of awareness, confidence and capacities, as well as of livelihood and welfare. The major breakthrough was the transfer of resources to SMBs for implementation and management of schemes.
Environmental impact. The environmental impact of ARCDP has been neutral or positive, but there are environmental problems concerned with the spread of shrimp farming, notably the contamination of groundwater in the sandy areas. On the other hand, afforestation of the dunes has resulted not only in improvements of soil and water but in a greater understanding among villagers of the importance of tree planting and its continuation after project closure.
Impact on institutions and policies. In terms of institutional development, the major efforts of the project were the building of management and technical capacities within the line departments, the establishment of an agricultural extension system to commune level, and the development of grassroots institutions. Institutional support was funded by UNDP, which reported major improvements in terms of knowledge, skills and effectiveness at all levels. Under the agricultural development component, trained extension workers were appointed in each commune, an important achievement. Some farmers, however, reported the performance of commune level extension as unsatisfactory and further investment in training is essential.
20. Sustainability. The sustainability of infrastructural improvements was not felt by the mission to be an issue, but there are concerns in other respects, notably regarding participatory approaches, as follows: (i) line agencies are not equipped to manage the dual responsibilities of technical support and the promotion of genuine ownership of schemes by the communities; (ii) the PRA conducted by the project were activity-specific and the required linkages between different components and programmes were lacking in project design; (iii) social mobilisation was utilised principally as a means of encouraging community involvement in planning and implementation rather than as an instrument of empowerment; (iv) in the three pilot schemes, the institutional structure was over-complex and paid insufficient attention to existing channels of administrative authority; (v) some self-management boards, for example in the sand dune fixation component, disappeared after the completion of the project; (vi) in 2002, UNOPS reported that only one-quarter of the savings and credit groups were still functioning.
With regard to training and capacity building, the mission made the following observations: (i) the building of the capacity of the technical departments will not result in a dynamic process unless ongoing training programmes are systematized within the departments; (ii) for all training courses, an archive of courses, materials and manuals must be built up; (iii) the usefulness of courses will be enhanced if their duration is increased and the content focuses more on practical field-based instruction; (iv) the utilization of ICT should be considered, so that digitized versions of the courses can be stored in project or line agency databases; (v) agricultural or aquacultural training is much less effective when the credit programme is inoperative.
Innovation and replicability. The most important achievement of the project in this respect was the transfer of the financial management of development funds to village level. The ‘model' SMBs have already been utilised by the districts as part of their own programmes. Similarly, the Water Users Groups set up under the ARCDP irrigation have been replicated by the province for all small-scale irrigation schemes throughout the province. A further innovation of the project was the practice of carrying out separate PRA exercises for groups of men and women. The provincial initiatives to continue the employment of commune-level extension workers after project closure should also be commended.
Performance of the project
Relevance of objectives. The ARCDP attempted to addressed the shortage of productive land by expanding the irrigated area; the remoteness of communities by building and upgrading new roads and markets, the declining income from in-shore fishing by providing alternative sources of income, the problems of under-employment by providing credit, training and expertise in relevant areas and the destructiveness of typhoons through its contribution to the afforestation of the dunes. In terms of the strategies of IFAD, the project followed the main recommendations of the 1996 Country Strategic Opportunities Paper (COSOP). The major design weaknesses related to the lack of integration between components, the formulation of the credit programme and the lack of poverty focus in the targeting of activities. Although poorer households benefited from the upgrading of roads, markets, irrigation schemes and domestic water supplies, they were not specifically targeted by the activities relating to increased agricultural production, animal husbandry and income generation.
Effectiveness. Assessment of the effectiveness of the project is bedevilled by the failure to set clear, measurable and time-oriented objectives for each component. The ‘reduction of poverty' is the overall intention of the project but an appropriate hierarchy of objectives is required. For example: in the sand fixation component, the objective must be the degree or rate of fixation of the dunes not the number of hectares planted; for roadbuilding, the objective might be defined in terms of the reduction in the cost or time of transportation not of the total length of upgraded roads; for the irrigation component, the area irrigated or the number of households affected could be set as the specific objectives. In some project documents, the objective at the component level was taken as the delivery of the budget. If the primary objective of the various components is seen in terms of physical or financial targets, all of the components apart from the credit component were ‘effective'. However, these objectives have not been matched with the overall objective of the reduction of poverty and food insecurity.
Efficiency. The major issue raised by the mission in this respect is whether existing institutions have been sufficiently utilised in the identification of the needs of villagers and in the planning and implementation of activities. The achievements of the Social Mobilisation model in three villages were substantial, but project specific structures have no proper legal status, their functioning is dependent on the approval of the CPC and they tend to be led by the incumbent village heads and commune officials. The model was imported from northern Pakistan, where it was designed to fill an institutional vacuum at the local level, but in Vietnam an administrative system is in place down to commune and village level and powerful mass organisations exist. Strengthening the existing institutions and involving them closely in project activities would be a more efficient, cheaper and more sustainable means to the desired ends.
The main reasons for the poor performance and ultimate scrapping of the rural credit component were the existence of cheaper sources of credit and the lack of incentives for the participating bank. Collateral-free credit was to be an integral part of ARCDP activities and, by serving to support beneficiaries for improved livestock and crop production and other income generation enterprises, was to be the primary means of targeting the poorest households. Because of collateral requirements and cumbersome procedures, alternative sources of credit remained beyond the reach of the poorer households. These problems were repeatedly raised by UNOPS supervision missions yet no effective action was taken until 2001. The fault may have lain partly with the infrequency of supervision missions, but there must also have been shortcomings in the communications between the project, the Cooperating Institution and IFAD.
The performance of partners
IFAD. The choice of Quang Binh province and the selection of activities were in line with the overall country strategy of IFAD and with the stated priorities of the Government. Project design can be criticised, however, for its weak poverty focus, its failure to insist on non-sectoral and objective PRA and the faulty design of the credit component. The most important innovation created by the social mobilisation model was the transfer of the management of development funds to village level. If the necessary regulatory framework can be established, this may prove to be the most radical of project impacts in institutional terms.
IFAD contributions to policy development are made indirectly through partners such as UNDP as well as through IFAD's reputation for innovation in the field. The Government acknowledged its debt to IFAD experience in its formulation of the CPRGS. However, the mission encountered surprising ignorance of IFAD even among senior officials of donor-funded programmes, and more needs to be done in developing synergies with other poverty projects through policy dialogue and partnership building.
The Government of Vietnam. The Government has been consistently supportive of the ARCDP. Cumbersome procedural requirements, for example for withdrawal applications, have been simplified. A gradual process of decentralization has been underway, and what is now required is a formal regulatory framework permitting the financial management of schemes at village-level. The provincial and district level departments were responsible for the replication of project models, particularly the O&M of irrigation schemes by beneficiaries, and the PPC ensured the continuation of the new agricultural extension system through the provincial budget.
UNDP. With its close ties with the government and its strong in-country presence, has a significant influence on developments in policy and approach, and IFAD has benefited not only from the technical assistance provided by UNDP but through its presence in key fora at central level. If UNDP is to be involved in supervision of IFAD projects, a possibility supported by the Government, a detailed assessment of UNDP's operational capacities will be required.
UNOPS. Carried out the supervision missions as stipulated but the infrequency of field visits, the lack of prompt follow-up and changes in personnel are constraints. Problematic issues such as credit, participatory processes and targeting were repeatedly highlighted in supervision reports, but remained unresolved for long periods. For more prompt and effective follow-up, changes in the supervision arrangements between IFAD and UNOPS are necessary.
Conclusions and recommendations
The province of Quang Binh had a very high rate of poverty at the start and this rate, although still unacceptably high, has fallen steeply. Given the size of the ARCDP investment (USD 14 million over six years) and the multiplicity of interventions in most communes, it may be assumed that much of this improvement is accountable to the IFAD-funded project. The multi-component, multi-sectoral approach seems correct in the circumstances, and the emphasis on agriculture, aquaculture, livestock, communications and markets was appropriate in terms of the overall objective of poverty reduction.
The mission concludes that a second phase of the project would constitute a constructive use of IFAD resources in Vietnam, on the following grounds: (a) the poverty rate is still high (around 20%) and there are still communities in remote inland areas and in the sandy zone of the coast suffering from food insecurity, lack of employment opportunities and basic amenities such as reliable water supplies; (b) many of the ARCDP interventions seem incomplete, especially in the sphere of capacity building and training, in which a start has been made but continuing efforts are necessary to consolidate the investments to date. The major issues requiring attention in the formulation of a second phase are listed below:
Greater integration between diverse activities. In one commune, project activities may have consisted of the building of a road and the expansion of the irrigated area; in another, a bridge, shrimp ponds and tree-plantation; in a third, savings and credit groups and the rehabilitation of the local market. These activities remained sectorally based and effectively separate. As a result, the ARCDP lacked a recognisable identity. The mission recommends that schools be used as the focal points for development activities in a given area;
The poverty focus of the project was weak. The main reasons for this were a lack of precision in project design, a tendency to predetermined activities in certain key components and the weakness of the credit component;
The objectives of the project were not clearly defined. For some components, the expenditure of the budget or the fulfilment of physical targets became the major criteria of performance; and
Project-specific institutions may not be sustainable. Alternative or parallel institutions are exotic, and the revised institutional arrangements may serve only to meet the conditions of the new investment and to act as the conduit for the funds. No criticism is implied of the aims of the new arrangements, but the mission has doubts about their sustainability. One of the model SMBs was disbanded on account of an administrative change, and the others, with their present complexity, may only survive if the levels of external investment continue.
The mission recommends that Phase II should develop active partnerships with ADB, particularly since ADB's Central Region Livelihood Improvement Project pursues similar objectives with the same target group and in the same province. Strong collaboration is encouraged in the area of information exchange, experience sharing and policy dialogue with the Government. The mission further recommends that the resources of IFAD Phase II should concentrate on the 12 communes of the sandy zone, the coastal communities of the river estuaries and islets and the poorer communes of the inland areas of Bo Trach, Quang Trach, Quang Ninh and Le Thuy.2 The identified coastal communes should be segmented into development areas and each area should be targeted with specific objectives. All activities in the area need to be integrated so as to lead to the achievement of the area objective set. The main emphasis should be on agricultural development and income diversification, facilitated by appropriate credit arrangements.
Infrastructure. The cost of upgrading roads in Phase I exceeded the planned cost by 80% and the cost of irrigation schemes by 35%. In future, these costs must be strictly controlled. Roadbuilding as a single, stand-alone intervention should be avoided. The cost of maintenance of unpaved roads is usually high and the project may consider tarring such roads from the outset.
Agricultural extension. The foundations for a commune-level extension service have been laid, but further training, as well as the careful screening of personnel in terms of motivation and suitability, is required. Given the diverse activities and large areas involved, specific and achievable aims must be set and monitored.
Rural Credit. The repeated recommendations made by UNOPs in this respect need to be examined and the design of the component thoroughly revised. The lessons learned from the IFAD-funded Ha Tinh project (HTRDP) can be applied. The mission recommends that there should be no haste to connect savings and credit groups with financial institutions. Financial discipline and management of intra-group loans are first necessary.
Sand dune fixation. In Phase II, the mission recommends that the process of afforestation be community-targeted, without the setting of an overall target in terms of hectarage. What is required is a zonal approach to match the situation and needs of each community, which would be responsible for planting and protection. It is also essential for sustainability that plantations be included under forestry protection schemes, to provide an incentive to villagers to protect the trees after project closure.
Aquaculture. The farming of shrimp has been out of reach for poor households because of the size of the initial investment. The initiatives of the DANIDA-funded programme within the Ministry of Fisheries should be examined, but aquacultural objectives should probably concentrate on small-scale processing and fish cage culture.
Community development fund. A key ingredient would be the setting up of genuinely flexible Community Development Funds, of which the investments would be owned and managed at village level. However, it is recommended that models of participatory development should utilize existing commune and village level organisations institutions, and the role of the CPC and other institutions should be formalized in this context. Community Facilitators (CFs) should be directly employed by the project to guide and support participatory development. A model for these CFs exists in the poorest 48 communes under the HTRDP.
Project organization. The mission considers the design of project organization more appropriate for strategic and policy making levels than for operational management. In Phase II, it may be preferable for the project directly to employ full-time technical staff rather than make part-time use of existing departments.
Management information system. A radical overhaul of the present arrangements is required. The management function and information needs at the various levels must be assessed, and an appropriate reporting system devised; a database for the PCU and the Technical Departments should be designed, with suitable access system and security mechanism. Reports should be digitized and electronically disseminated. A wide area network should be set up, using the existing telephone network or by establishing a private virtual network on the internet.
Institutional support. Further investment is required in the capacity building and outreach of the various line agencies, particularly with regard to the extension services for agriculture and aquaculture. The new environmental department (DONRE) urgently requires investment in terms of equipment and skills.
Supervision. More prompt follow-up is required on the recommendations of missions, and some contractual changes may be necessary between IFAD and UNOPS.
Technical assistance. Appointments under TA should be made on a short-term basis with very specific conditions in terms of tasks and objectives. Given the availability of expertise available within the country, local consultants should be preferred wherever possible.
Policy dialogue. The Government of Vietnam and the other donors recognize IFAD experience, expertise and knowledge of poverty alleviation, but there is a clear need for IFAD directly to involve itself in the development of national policies and strategies and of synergies with other poverty alleviation programmes through the instruments of policy dialogue, coordination and partnership building. IFAD should give further consideration to the vexed issue of maintaining some form of in-country presence.
1/ The composition of the Interim Evaluation Mission was as follows: Roger Norman, Team Leader and environmentalist; Ravi Raina, infrastructure consultant; Qaim Shah, agricultural and institutional consultant; Tran Thi Tram Anh, gender and participation consultant; Sarath Mananwatte, Economist. The Mission included as observers national counterparts from the Provincial Project Coordination Unit and District cadres. Ashwani Muthoo, IFAD Senior Evaluation Officer, joined the Mission in its final week. The National Wrap-up Meeting was chaired by the Vice Director General of the Department of Agricultural Economy, Ministry of Planning and Investment and attended by Mattia Prayer Galletti, the outgoing IFAD Country Programme Manager (CPM), Ms Atsuko Toda, the designate CPM and members of the formulation mission for the proposed follow-on project as observers, as well as key decision makers at provincial level and from the concerned ministries and partner agencies.
Development Project for Indigenous and Afro-Ecuadorian Peoples (2004)
The design of the Development Project for Indigenous and Afro-Ecuadorian Peoples (PRODEPINE) ties in with the government strategy for sustainable development that calls for preservation of the identity of the 12 nationalities and peoples of Ecuador in the country's three regions: the coastal areas, the Sierra, and the Amazon. The project covers 19 of Ecuador's 22 provinces, 108 of the 213 cantons, and 434 of the 788 rural parishes. The population covered by the project is approximately 1 440 000, including 1 346 000 indigenous and 94 000 Afro-Ecuadorians in 4 748 communities, surpassing the original target by 57%. Project results indicate that resources are concentrated in the Sierra region, where most of the indigenous population lives, represented by the Kichwa de la Sierra nationality.
PRODEPINE was co-financed by IFAD (USD 15 million), the World Bank (USD 25 million) and the Government of Ecuador (USD 10 million). The World Bank completed its part of the operation in December 2002. Investments as of 30 April 2004 stood at USD 14 213 50, leaving a balance of USD 1 349 048 to be disbursed by 30 June 2004.
The objectives set for PRODEPINE have been maintained throughout the project:
strengthen the technical, administrative, and managerial capacity of indigenous and Afro-Ecuadorian communities at the local, regional, and national levels in order to promote more effective participation in official policy-setting and improve the provision of economic services to those communities;
achieve the democratic integration of indigenous and black peoples, incorporating their own development vision and leveraging their current resources as well as their human and social capital;
systematize and improve access for project beneficiaries to land and water resources;
increase capacity at local levels to enable community members to take part in identifying, designing, and carrying out productive and social infrastructure projects;
leverage financial resources for productive investments for rural communities, families and individuals; and
strengthen state institutions to create adequate capacity for policy-setting, planning and coordination of activities designed for the target population.
The four project components have not changed, except the rural financial services (RFS) component (fully funded by IFAD) which was redesigned twice to improve conditions for implementation. The components are as follows:
- Component 1: Institutional strengthening for second- and third-tier indigenous and Afro-Ecuadorian organizations
- Component 2: Support for regularization of land and water rights
- Component 3: Rural investment and credit
- Component 4: Institutional strengthening of the Council for Development of Nationalities and Peoples of Ecuador (CODENPE) and the Development Corporation (CODAE)
Political and institutional changes during implementation. PRODEPINE has been subject to both external and internal political changes. Firstly, the project was carried out during a period of political instability, with three consecutive government administrations. Secondly, the creation of CODENPE as the successor to the National Planning and Development Council for Indigenous and Black Peoples (COMPLADEIN) led to the need to refocus the project on the concept of nationalities and peoples. Despite the tensions generated by these changes, implementation continued successfully. CODENPE was effective as the national counterpart organization representing indigenous peoples. As to the participation of the National and Afro-Ecuadorian Council (CNA) in representing Afro-Ecuadorians, the Council was important as the antecedent to the Council for Afro-Ecuadorian Development (CODAE) whose operational mechanisms are still being set up.
Changes to design during implementation. Both minor and major adjustments were made. Minor design adjustments were made in connection with the "collective rights" of indigenous and Afro-Ecuadorian nationalities and peoples as stipulated in the new Constitution of 1998. More substantial changes were made to the RFS component in response to external and internal factors such as: (i) the banking crisis of 1999-2001; (ii) communication problems between PRODEPINE and IFAD concerning the component's set-up; and (iii) internal organizational and strategy-related matters within PRODEPINE. Implementation of all the components began in late 1998, except for the RFS sub-component which experienced a delay of two and a half years, causing it to be isolated from the other components and leading to disconnect with financing from the World Bank.
Financial performance of investment of IFAD funds. Following adjustments to exchange rates as a result of the polarization of Ecuador's economy, the amount of IFAD Loan 464-EC stands at USD 14 332 000, financing the following: credit for land purchases, water consulting, rural investments for public benefit (sub-project financing), rural investments for private benefit (private sector), institutional strengthening for CODENPE and CODAE (operating expenses), technical assistance for financial intermediaries, goods for financial intermediaries. The mission recommended a second extension to March 2005 to allow adequate time to disburse the remaining USD 5.2 million of IFAD's funding.
Results of Component 1: Institutional Strengthening for Second-Tier and Third-Tier Organizations (STOs and TTOs). This component, the backbone of the project, is fully funded by the World Bank. The component has strengthened 241 STOs and TTOs in the implementation of 212 local development plans (LDPs) for indigenous and Afro-Ecuadorian people (19% more than planned), benefiting 1 345 500 indigenous and 93 600 Afro-Ecuadorians. The human resources training sub-component has shown positive results. PRODEPINE has financed scholarships for 69 trainees and 335 high-school graduates, as well as 646 local technicians, of whom 43% were women. The cultural heritage sub-component funded 10 publications, videos, seminars, etc. The component includes support for ten indigenous peoples fighting for survival and threatened with extinction: Siona, Secoya, Cofán, Huao, Shiviar, Zápara, Awá, Epera, Chachi and Tsáchila.
Results of Component 2: Support for the Regularization of Land and Water Rights. Under this component, IFAD financed the land purchase sub-component for USD 345 455 (3.7% of IFAD investment), with 634 ha of land purchased benefiting 185 families in eight organizations of the Sierra region (Tungurahua, Imbabura, Pichincha, Cotopaxi, Bolívar). Support for the preparation of 71 land management plans is envisaged by project completion. Land legalization was funded by the World Bank. For land belonging to the National Agricultural Development Institute (INDA), 105 596 ha of land was legalized by means of titles issued to 55 target communities (1 832 beneficiary families). For land belonging to the Environment Ministry, 17 084 ha of land was legalized, benefiting 16 communities (273 families). For the water component, diagnostic assessments were carried out for 458 community irrigation systems (2 647 km) belonging to 37 194 beneficiary families in 793 communities.
Results of Component 3: Rural Investments for Public and Private Benefit. This component includes two sub-components: (i) investments for public benefit or sub-projects (38% IFAD financing, 62% World Bank financing); and (ii) RFS investments for private benefit (100% financed by IFAD).
IFAD investment in the sub-project sub-component was USD 5 204 563: USD 4 467 060 for sub-projects; USD 395 740 for pre-investment; USD 319 720 for technical training; and USD 22 043 for monitoring of sub-projects. Total investment in the component benefited 62 644 families in 103 cantons, 57% more than planned. The sub-projects reached 26% of all indigenous and Afro-Ecuadorian peoples. They responded to 33% of requests from grass-roots organizations and involved 1 559 STOs and TTOs. Most of the investment went to the Kichwa nationality (401 sub-projects and USD 7 789 178 or 64.77% of the total investment made by PRODEPINE, covering 270 409 inhabitants). In second place was investment destined for Afro-descendents (71 projects financed with USD 1 708 407 or 14.21% of total investment). Third was investment for the Shuar people (90 sub-projects and USD 1 179 564 or 9.81%), followed by the Achuar nationality (62 projects and USD 497 221 or 4.13% of total investment). Just 7.08% of the total investment, or USD 850 819, went to initiatives initiated by the tribal peoples (Awa, Chachi, Epera, Huaorani, Shiwiar, Tsachila and Zápara).
The RFS sub-component includes the Second-Tier Credit Union Programme (CUP) and the cajas solidarias Village Banking Programme. Rural investments for private benefit totalled USD 3 166 761 or 34.1% of the IFAD investment. As of April 2003, the credit component had reached 62% of the target number of beneficiaries established in the original project design, 58% of the lending target and 85% of contributions to cajas solidarias. Despite the delay in implementation, the evaluation team found that the RFS sub-component showed major achievements in implementation. The mission feels that this sub-component contributed to expanding the supply of micro-financing to low-income sectors that include indigenous and, to a lesser extent, black population groups.
Within the CUP Programme, USD 2.2 million has been disbursed from the IFAD funding: USD 270 000 in member contributions (cash reserves) and USD 371 000 in funding for the 17 credit unions, for a total of USD 2.9 million (USD 959 per beneficiary), distributed to 3 000 families. There is a certain concentration of RFS sub-component activities in the regions of the Sierra (77.2%: 42% Central, 22% South, 13% North) attributable to the institutional and financial capacity of local agencies to take on responsibility for intermediating PRODEPINE funding, followed by the Amazon (11.9%) and Coast (10.9%) regions.
As regards cajas solidarias, close to 15 000 women were served through 626 cajas and total contributions of seed capital reached USD 425 000. Total investment by PRODEPINE was USD 558 000, i.e. an average of USD 912 per caja. The local contribution (mandatory savings to set up a caja) was USD 211 per caja (USD 129 000), which implies that the women have achieved a level of capitalization of 23% over an average period of two years. This is considered a satisfactory result. The cajas are also concentrated in the Sierra region (65%), with the regional Central Sierra office accounting for 48.5%, followed by the Coast with 21.3%, then Amazon with 13.2%.
Results of Component 4: Strengthening of COMPLADEIN (currently CODENPE and CODAE). This component was classed as "satisfactory" by the World Bank project completion report (PCR). Although most of the funding for this component came from the World Bank, IFAD invested USD 583 533, representing 6.2% of the IFAD investment as of 30 April 2003.
Generally speaking, the project completion reports drafted by the World Bank and by CODENPE-PRODEPINE have evaluated performance by the four project components as "satisfactory" according to the project completion evaluation parameters of the World Bank and the project executing unit.
Relevance of objectives. The evaluation indicates that the PRODEPINE objectives are highly relevant in responding to the priorities established in diagnostic assessments of indigenous and Afro-Ecuadorian peoples drawn up during project preparation.
Effectiveness. PRODEPINE has not, in four years, changed living conditions for the 3 695 communities concerned, nor did it set out to do so. However, it has made a very effective contribution to setting up a participatory national planning structure through STOs and TTOs. Thanks to this structure, which combines state institutions with indigenous and black civil society organizations, operating conditions today are much more favourable to implementing such policy. The effectiveness of PRODEPINE has made it possible to visualize, as a natural next step, adjustments that can be made to processes in order to better serve the Coast and Amazon regions.
With respect to strengthening of indigenous and Afro-Ecuadorian institutions, PRODEPINE has been very effective in implementing the state policy of CODENPE (and, to a lesser extent, CODAE) to consolidate and empower indigenous and Afro-Ecuadorian organizations and to carry out 152 development plans for STOs and TTOs. This kind of policy has never before been implemented in the history of Ecuador and sets a standard for the rest of Latin America. Nevertheless, the project beneficiaries feel that there are administrative and organizational aspects of implementation that diminish the component's effectiveness. These aspects have to do with political pressures, on the one hand, and the accountability of organizations to communities and the State, on the other.
The public investment component has effectively responded to 95% of the priority needs of communities.
As to the RFS sub-component, those interviewed corroborate a concern with the line of credit granted through local financial agencies (LFAs). The RFS Division of PRODEPINE considers the transfer of funds to LFAs as efficient and effective. The evaluation, however, indicates that the project's role of facilitating development of the rural financial sector to serve the target population through LFAs is still incipient.
Efficiency. PRODEPINE was approved by the IFAD Executive Board in December 1997 and entered into effect in November 1998, at which point all components began operations except for rural finance and land purchase (both funded by IFAD), which commenced later. Generally speaking, it can be said that PRODEPINE has been moderately efficient in establishing mechanisms and instruments to promote lasting development processes. Through PRODEPINE, CODENPE now has effective instruments to stimulate indigenous development in Ecuador. For example, there is greater knowledge of strategic plans within CODENPE and CODAE. At the level of each nationality there are now participatory tools and a methodology2, as well as instruments to build an organizational and operating structure. It is urgent that these instruments be adapted to the regional environment in order to make them more efficient.
Regarding the public investment component, PRODEPINE has very efficiently put in place processes within indigenous and Afro-Ecuadorian organizations, but not in the communities themselves. The field evaluation found that community participation had been limited to setting project priorities and their labour contributions and that training of communities to participate effectively in monitoring and implementation, or social, administrative and financial control of their projects was still incipient.
With respect to RFS, according to those concerned, the cajas solidarias sub-component has been efficient in bringing small loans to the indigenous rural population. As to RFS through CUP, the efficiency of these has been questioned based on the following observations: (i) the interest rate charged to credit unions by the project is three to four percentage points below the market rate, which attracts larger LFAs with no liquidity problems into the financial system. Also, larger credit unions receive a larger proportion of the subsidy and are rewarded with a discount of one point as a performance incentive. (ii) PRODEPINE has selected credit unions with good financial performance, which significantly limits financial risk; and (iii) use of the Bank Watch Rating excludes all LFAs with less-than-good financial performance, even if they have some potential, for both the credit and the strengthening programmes. This policy has had a particular impact on regional offices where the supply of financing has been limited (Amazon and Coast).
The impact on the country's three regions largely reflects the amount invested. The greatest beneficiaries of this first PRODEPINE have been better organized and more established STOs and TTOs, and these organizations are located in the Sierra. There is a pronounced need for differentiated strategies for the Coast and Amazon.
The impact on material and financial resources is reflected in: (i) the institutional strengthening of CODENPE, CODAE, STOs and TTOs in administrative procedures for management personnel and in office equipment; (ii) the family and community environment of beneficiaries through the social infrastructure sub-projects that make up the community assets (for example: water for human consumption and irrigation, electricity, schools, health centres, day care centres) and productive sub-projects that have raised the value of land holdings and, according to the testimony of beneficiaries interviewed, have also raised family incomes; (iii) the impact of cajas solidarias on rural communities that would otherwise have no access to credit; and (iv) the appreciation of land purchased or legalized and cultivated.
The impact on human resources, by facilitating new skills, is visible mainly among the leaders of indigenous organizations at the national level, in the following areas: project design, formulation, implementation, monitoring and evaluation, as well as support for accounting and administrative management, all of which has improved their leadership skills. It is important to recognize, however, that this knowledge does not always trickle down throughout the organization. Training for community human resources in project management and implementation, under the STOs and TTOs, was very limited. On the other hand, it is difficult to evaluate the impact of RFS on credit unions, since other agencies such as Swiss Contact, the Inter-American Development Bank (IDB, and the World Organization of Credit Unions (WOCCU) have also invested in the human resources of these credit unions.
The project's impact on social capital and empowering capacity for action by organizations is greater than the impact on social capital of beneficiary communities, which depended largely on each STO or TTO participating, as well as the mechanisms used to put in place the capacity for self-management, monitoring, and social auditing. On the other hand, PRODEPINE supported an increase in the social capital of local networks of paralegals facilitating land arbitration.
Impact on food security was not an explicit project objective. However, families interviewed identified this as one of the project's positive impacts. Even some sub-projects such as schoolrooms, child-feeding centres, shelters and day care centres, have benefited food security by forging strategic alliances with state and private programmes that promote school breakfast and lunch. The land award projects also have an impact on family food security since families then devote time to cultivating and looking after the land.
PRODEPINE has supported the preservation and conservation of the environment, particularly fragile ecosystems such as wild land, andean and tropical forest. However, it is not clear that the management plans prepared are being implemented, since the communities require further training in order to do so.
The impact on institutions, policy, and regulatory frameworks has been significant in supporting the formulation of development plans for CODENPE and CODAE and local development plans (LDPs). Most of the targets have been reached: (i) The "nationalities and peoples act" placed before Congress for adoption; (ii) the "act recognizing traditional health systems"; and (iii) several acts protecting indigenous cultures, childhood and adolescence, judicial administration, bilingual education and civil participation. In order to support the decentralized administration of CODENPE, the component set up and equipped seven regional offices. The project was successful in preparing and presenting to Congress the special Ecuador "biodiversity conservation and sustainable use act".
Impact on women. Since there is no gender strategy included in the project, or any systematic monitoring in the Rikuk Pach information system of participation by men and women in the various components, the gender impact of the project is not clear. As of mid-term, 20 gender promoters had been trained under the cultural heritage sub-component. The cajas solidarias sub-component is devoted exclusively to women. Although the cajas are small in size, their potential social impact is great. Land titling and transfer has helped to empower female heads of household and improve their self-esteem. Evidence was found that the infrastructure, cajas solidarias and scholarships sub-components have empowered women and improved their position within the family and society.
Project sustainability and replicability. At the moment, the project depends on external funding. The national contribution is approximately 10%; however, as of April 2003, the Government of Ecuador had disbursed just 35% of its contribution. The economic and financial sustainability of the infrastructure projects is questionable, since the communities have not been trained to maintain them. The replicability of the project is seen by beneficiaries as an urgent need, since this is the only project of its kind in the country.
Other kinds of impact on poverty. One of the project's greatest achievements is that it promotes the recognition of the political, legal, and physical space that the nationalities and peoples need to occupy in order to overcome their social and economic constraints.
Overall impact evaluation. It can be said that the repercussions of PRODEPINE are significant both in the target population and in the mixed-race population, in terms of: (i) socio-political impact in the country; (ii) impact by providing a window of opportunity to the target population; and (iii) impact by spearheading policy implementation by CODENPE. PRODEPINE designed several participatory instruments and methodologies to evaluate impact, under the Regional Programme for Strengthening Regional Evaluation Capacity (PREVAL) associated with IFAD.
Performance by cooperation agencies IFAD and the World Bank. The mid-term evaluation indicates that coordination between co-financiers IFAD and the World Bank is poor. This may have had some influence on the delay in implementation of the RFS sub-component. The World Bank administered IFAD funds up until 2002, as well as semi-annual supervision exercises of the components co-financed. IFAD took part in some of these. The category reallocation took place in 2003. There is a widespread perception that IFAD has been "absent" at critical decision points since July 2001. Whereas the World Bank had completed its operation (USD 25 million), IFAD had disbursed just USD 9.3 million by April 2003, leaving a balance of USD 5.2 million.
Performance by the Government of Ecuador and its agencies. The project has been successful in supporting CODENPE in setting policies and in promoting their application in the national line ministries, for example, the National Intercultural Bilingual Education Directorate (DINEIB) and the National Indigenous Health Directorate3. The evaluation verified a total appropriation of the project by CODENPE. As to national counterpart funds, given the serious economic and financial crisis, the government was able to cover only 35% of its contribution, which served to defray taxes and pari passu expenses. This had an adverse impact on the components that depend on counterpart funding, such as the land purchase and titling and water rights studies sub-components. However, the rural investment for public benefit component, 36% of which was financed by IFAD, commenced on schedule thanks to contributions from beneficiaries. Implementation of the land purchase component was affected by the elimination of the Ecuadorian Forestry Institute for Nature and Wildlife and its replacement by the Ministry of Environment as the governing oversight agency for sustainable environmental management. This change caused the delay in commencing operations under the component. Performance by the Ecuadorian fund Populorum Progressio (FEPP) and INDA under the land purchase sub-component financed by IFAD is considered "satisfactory".
Performance by NGOs and grass-roots organizations. The evaluation concludes that the performance of STOs and TTOs was "satisfactory" within the parameters set by the project, but it is not clear that it was satisfactory from the point of view of community participation and the transfer of power to the communities. Both the annual external audits and the communities themselves have indicated that much remains to be done in order to achieve transparent, participatory management.
Generally speaking, PRODEPINE is considered a highly replicable and successful project, both because of its relevance within the socio-economic context in Ecuador and because of its effectiveness in setting up an operating structure at the national level. Above all, this was achieved in the midst of a serious economic crisis, social upheaval, and far-reaching institutional change.
PRODEPINE was created to implement state policy through CODENPE (and today, CODAE) to benefit indigenous and Afro-Ecuadorian peoples at the national level. Indeed, the implementation of this policy has now set a benchmark for all of Latin America.
Since the project did not establish a baseline, it is difficult to determine the changes in living conditions for the 3 695 communities surveyed; however, it can be said that a decentralized operating structure has been introduced in these communities that translates into participatory planning through the country's STOs and TTOs. Thanks to this structure which brings together both state institutions and indigenous and black civil organizations, operating conditions are today much more favourable and positive for implementing such policy through a second World Bank loan and perhaps a second IFAD loan as well.
Development with identity. PRODEPINE was designed as a development project with identity. For administrative reasons, the same administrative and operating procedures were applied in all three regions of the country, without taking into account that they are culturally, climatically and ethnically different. There is a widespread perception that PRODEPINE was designed for the Sierra, where most of the country's indigenous people live.
Regionalization and differentiated treatment. It is important to adapt processes to the ethno-cultural, climatic and geographical variables that call for a differentiated approach in the three regions: Coast, Sierra and Amazon. This means different per capita allocations in the three regions. Interculturalization is achieved as a result of peoples' development with their own identities – what does this mean?
Monitoring. The national-level information system known as Rikuk Pacha is an efficient reference system that records key information for the administrative monitoring of the project. However, this information is not sufficient to monitor indicators of social processes for poverty reduction, or to measure their results.
Gender. The only component with a gender strategy is the cajas solidarias sub-component which is focused on women, with loans used mainly for productive activities. Participation by women as end users of financial services has been satisfactory, with more than 15 000 women having benefited from the services offered by the cajas solidarias.
Community participation, accountability, and social audits. According to the annual audits of PRODEPINE, a significant impasse occurred over justifications for expenses and accountability by the STOs and TTOs to regional offices, since rigorous accountability is new to these organizations. Nevertheless, marked progress has been observed. Also, social auditing of STOs and TTOs for communities and the empowerment of communities for self-management are still embryonic.
Participation by STOs and TTOs and role of communities. The STOs and TTOs have received significant institutional strengthening to support communities in drawing up local development plans (LDPs). The STOs are the agencies that administer sub-projects. The field evaluation verified that the administrative capacity of STOs has remained with their leadership and has not trickled down to the remaining STOs or to member communities.
Institutional strengthening. This component strengthened 241 STOs and TTOs in the implementation of 212 indigenous and Afro-Ecuadorian LDPs, benefiting 1 345 500 indigenous and 93 600 Afro-Ecuadorians, as well as supplying office equipment.
Lack of baselines. The organizational indicators for the project were built gradually during execution of the consulting report by the Heifer Foundation. The project began without first establishing a baseline.
Coverage. Although it is true that those who have benefited most from this first PRODEPINE are the best organized and well established STOs and TTOs in the Sierra region, it is also true that PRODEPINE has promoted the strengthening of STOs and TTOs throughout the country, and has created new organizations that otherwise would not have been formed.
Dependency on PRODEPINE. PRODEPINE has been very well received by its beneficiaries. In some communities visited, it is the only project they have been exposed to with a decentralized, culturally appropriate strategy. The mission would like to raise an alert that this could create a dependency on PRODEPINE. Self-management appears to have remained in second place. The strengthening of STOs and TTOs is a process that calls for long-term monitoring and follow-up by CODENPE.
Land purchase and water rights sub-component. Of the IFAD investment, 3.7% (USD 345 455) was earmarked for land purchases with support from FEPP. Under the land purchase component financed by IFAD, 634 purchases have been made, benefiting eight organizations in the Sierra region. It is expected that upon project completion, support will have been provided for the preparation of 71 management plans, which is a requirement for obtaining property title. There is some question, however, as to whether these plans are actually being implemented. As to the water sub-component, the project supported a study of community irrigation in 793 communities and 40 STOs, and 458 irrigation systems underwent a diagnostic assessment.
Rural financial services sub-component. As of April 2003, the RFS component had disbursed a total of USD 3 166 761, 34.1% of the IFAD investment. Despite the delay in start-up, the RFS sub-component has made major strides in implementation. This evaluation concludes that the sub-component has contributed to expanding the supply of microfinance to low-income sectors among indigenous and, to a lesser extent, black populations. Through the cajas solidarias, a process has been established that stimulates the creation of effective demand for financial services among low-income population groups. The delay in commencing the RFS sub-component was attributable to factors such as the financial crisis at the time of project start-up, communication problems between the project and IFAD, and the sub-component's isolation within the project itself. During the first two years of implementation, the sub-component was not able to come up with an adequate response to address these problems.
Public investment subprojects. This sub-component was financed by IFAD (38%) and by the World Bank (62%). The sub-component financed 661 sub-projects, benefiting 62 644 families in 103 cantons, i.e., 57% more than planned. The sub-projects reached 26% of all indigenous and Afro-Ecuadorian peoples. They responded to 33% of requests from grass-roots organizations (1 582 organizations). Afro-Ecuadorians received 16% of the investment for 71 projects, equivalent to USD 1 708 407; and the National Amazonian Federation of Campesino and Indigenous Organizations (FENOCIN) received 13%, equivalent to USD 497 221, for 62 projects. The LDPs focus on generating outputs rather than on cultural revitalization processes or long-term organizational strengthening. They are prepared under a fundamentally economic and organizational approach.
To ensure the successful completion of PRODEPINE with IFAD financing, and in the event of new financing, the evaluation mission recommends the following:
Extension of completion date. Given that USD 9.3 million of the investment was disbursed over four years, and that there is a balance remaining of USD 5.1 million to be disbursed by March 2004 (including one extension), it is recommended that the completion date be extended to March 2005 so that the project may be completed satisfactorily. If IFAD decides to invest in a new project, it is recommended that preparations begin right away to avoid slippage, and that week-long missions take place twice a year.
IFAD presence. It is recommended that IFAD maintain more of a presence in supervision during the execution of the remaining 35% of Loan 464-EC, particularly since the World Bank has completed its PRODEPINE I operation. In the event of a new IFAD operation, it is recommended that a more adequate monitoring and supervision plan be drawn up.
Institutionalization of PRODEPINE as a government programme for indigenous and Afro-Ecuadorian peoples. Since PRODEPINE is a project that operates under CODENPE-CODAE policy for strengthening indigenous and black people, it is recommended that it become a government programme rather than a project. It is recommended that CODAE be strengthened as well.
Promote regionalized strategies. IFAD has shown an interest in "having differentiated mechanisms for different people, and particularly for Amazon peoples as opposed to highland Andean indigenous campesinos4". The PRODEPINE regional offices have expressed a need to adapt processes to regional realities. The northern Coast region would like to have an African PRODEPINE, the Amazon region an Amazon PRODEPINE.
Long-term institutionalized decentralization. Given the high degree of decentralization of public services in municipalities, the evaluators suggest that the PRODEPINE regional offices be brought closer, operationally speaking, to municipal governments, so that LDPs are not developed for PRODEPINE alone but rather include all critical factors for development (health, water and sanitation, education, etc.) in cooperation with all public and private agencies at the municipal level.
Interculturalization. It is recommended that the indigenous cultural indicators prepared by the Integrated System of Social Indicators for Ecuador (SIISE) be adapted to the System of Social Indicators for Nationalities and Peoples of Ecuador (SIDENPE), and integrated within the PRODEPINE II monitoring system. "Informed consultation" on the regional elements of "interculturalization" and intercultural adaptation of processes to regions should be promoted.
Continue focusing on critical factors of interculturalization, gender and community participation. Strengthen, within the Rikuk Pacha system, gender and cultural variables differentiated for all three of the country's regions. Ensure that a baseline exists at the outset of the next project.
Monitoring processes and results. Update the Rikuk Pacha system so that it reflects not only administrative processes but also processes and results that have a social impact, and train communities to carry out "participatory monitoring" of changes. Use evaluation methodologies and instruments prepared under the project itself (i.e. the methodologies of RURALINVEST, social capital measurement, LDP preparation, environmental monitoring and assessment, and rural development project evaluation).
Take strengthening to communities (what does this mean?). Under PRODEPINE I, institutional strengthening was targeted to CODENPE-CODAE and to STOs and TTOs, and to a lesser degree to beneficiary communities. It is recommended that communities be trained to: (i) self-manage, design and execute their own projects; and (ii) supervise and control the administration of funds by STOs and TTOs by "generating the budget in a participatory manner", through "participatory monitoring" and "community implementation". STOs and TTOs must be capable of administering funds and rendering accounts, but communities need to have greater influence in order to appropriate the implementation and maintenance of their own projects.
Introduce an annual update system for LDPs for communities. The LDPs are live documents and need to be updated accordingly. Those found date back to 1998. It is recommended that they be updated annually.
Community contracting. Subcontract project administration, training, etc. to NGOs and indigenous associations, in order to provide opportunities for professional development.
Solidify strategic alliances with other programmes both at the state level and with civil society – for example, the Urban Development and Housing Ministry (MIDUVI) - Housing, Water and Sanitation5, the Social Investment Fund of Ecuador (FISE)6, the Intercultural Bilingual Programme of the Ministry of Education (ME), the Department of Indigenous Health of the Public Health Ministry and others – to coordinate efforts directed at the target population.
In the event that further IFAD financing is granted, set up an Inter-Institutional Consultative Committee to design (with economies of scale) and coordinate policies and optimize results. The Committee could comprise: (i) CODENPE and CODAE; (ii) cooperating agencies, including the United Nations Food and Agriculture Organization (FAO), United Nations Development Programme (UNDP), the Regional Programme for Strengthening Regional Evaluation Capacity (PREVAL) and the World Bank; (iii) public institutions (Ministries of Health, Education, Agriculture, Finance, MIDUVI, PRAGUAS); and (iv) grass-roots indigenous organizations.
Land component. Follow-up is recommended for land development plans covering land purchased or legalized through PRODEPINE. At the moment, the implementation of these plans is the responsibility of the communities, with support from STOs. It is imperative that a programme such as PRODEPINE continue to cover demand for legalization and purchase of ancestral territories in all three regions, but most emphatically in the Amazon region where a diagnostic assessment has found 1 294 759 ha of land remaining to be legalized.
Water component. It is recommended that a study of irrigation be carried out, incorporating the needs of 70 STOs and 726 irrigation systems for 1 368 communities, to be financed with other funds under a subsequent phase of PRODEPINE.
RFS component. The mission's main concern with respect to RFS is the lack of consolidation. The sub-component is halfway towards achieving its objectives: "Increasing access by indigenous and black communities to financial resources that will enable them to fund productive activities and facilitate the establishment of efficient and sustainable financial systems for the target population". If the sub-component's activities were terminated at this time, it is possible that indigenous and black populations would lose the opportunity to access rural financial services. Based on the objective of this sub-component, strategic elements are suggested to develop the RFS:
Sector coordination: It is recommended that technical staff from PRODEPINE regional offices play a more active role as facilitators of local processes, coordinating their activities and forging strategic alliances with organizations working in the same sector;
Regional strategy: Given the high degree of heterogeneity of the local environments in which the project operates, it is recommended that three different overall strategies be developed in the context of each region (Coast, Sierra, and Amazon). Necessary steps for each strategy are discussed in detail in Appendix 3:
- Link cajas solidarias to LFAs. Facilitate connections: (a) between LFAs and (independent) cajas solidarias so that the former grant loans to the latter and these in turn on-lend to their members; or (b) between cajas solidarias to improve the financial capacity of small cajas.
- Assist LFAs in reducing their scale, i.e. include the PRODEPINE target group among their traditional clientele. Instruments for serving the project's target population must be transferred to LFAs.
- Professionalize LFAs with financial products directed to the target population. This strategy calls for qualitative changes so that an institution that is already serving the target population may do so in a more efficient, more stable and more professional way.
Credit fund strategy. The current two-tier model is considered adequate, and on this basis several elements are proposed for the second phase. It is suggested that responsibility for the fund be assumed by PRODEPINE II. This option would allow a technical unit to be financed at the national and regional levels, which would in turn allow for continuity in following up on agreements with LFAs and portfolio management. The fund would be administered by PRODEPINE. CODENPE would sit on the fund's board of directors. A credit committee would make final decisions on matters concerning lending policy. Income from the second-tier fund should cover the technical unit's operating and financial expenses, which is currently not the case. The technical unit in the post-project phase should be small (two or three people) with operating costs of not more than 5% of the loan portfolio (in 2002, they were 5.5%). If necessary, the current subsidy (3-4%) to LFAs would be cut and the interest rate raised.
1/ The Mission was composed of Ms. Sra. Ximena Traa-Valarezo, Anthropologist; Mr. Arie Sanders, Specialist in Rural Finance Sector; Mr. Vincent Brackelaire, Sociologist and Ms. Consuelo Rojas, Economist Spezialized in Rural Development. The misión counts on ample experience in evaluation of projects targeted to indegenous peoples. Ms. Katharina Kayser, Evaluation Officer, IFAD, participated during the first week of field work. Project Officials from PRODEPINE, both the Central and the Regional Offices, collaborated with the evaluation team without interfering in meetings, interviews or focus groups meetings in the field.
3/ Another two projects implemented by CONDENPE are the Cotopaxi Development Project (PRODECO) financed by the European Union and the Indigenous Alternative Municipalities Strengthening Project (FORMIA) financed by the Spanish International Cooperation Agency (AECI). All three projects (PRODEPINE, PRODECO Y FORMIA) are considered to be executing units of CODENPE.
Root and Tuber Improvement Programme (RTIP) (2004)
The Root and Tuber Improvement Programme, (RTIP) became effective in January 1999 and is scheduled to close in December 2004. IFAD and the Government of Ghana are considering a second phase for this project. This evaluation of RTIP was undertaken at the request of the IFAD Africa I Division as a precondition for the formulation of any second phase project. It took place over the period May – November 2003. The evaluation team conducted fieldwork in late July and early August, including visits to 11 districts and meetings with stakeholders in areas of RTIP activities as well as in Accra, Kumasi and Rome. Further, it drew upon a Beneficiary Assessment Study (BAS), commissioned by RTIP1.
The evaluation team used the IFAD Office of Evaluation Methodological Framework for Project Evaluation that emphasises consultation and participation of partners with the expectation that the knowledge they acquire will contribute to their improved performance in future. For this, the Core Learning Partnership1 was established to guide the work. This group reviewed and cleared an Approach Paper that outlined the evaluation methodology, key questions, and calendar.2 It held briefings and de-briefings with the evaluation team prior to and following fieldwork. It will review the draft evaluation report and meet to conclude an agreement among partners, known as the Agreement at Completion Point (ACP).
The project rationale justified an RTIP focus on root and tuber crops because of their importance in household food security, because they are grown by the poorest segment of the population, because their development would help diversify agricultural sector development, and because supporting them would channel more agricultural sector resources to the smallholders who produce, process and market them. RTIP was designed to be a nationwide project, referred to as a programme that targeted some 720 000 resource poor farmers, with priority to be given to women.
The overall objective of RTIP was to enhance food security and improve incomes of resource poor farmers. Originally this was to be done through five components: i) multiplication and distribution of improved material ii) integrated pest management; iii) adaptive research; iv) community support and mobilisation, and, v) programme management and coordination. A sixth component for post-production and marketing was added subsequent to a project start-up workshop in August 1998. The Ministry of Food and Agriculture, Crop Services Division, was assigned responsibility for RTIP implementation and the World Bank was appointed as the cooperating institution for project supervision and implementation support. Numerous research institutions, universities, non-governmental organisations and other agencies were assigned responsibilities for implementation. Liaisons with the International Institute for Tropical Agriculture (IITA) and with other IFAD projects in Ghana were also important to project implementation.
One major policy change, the President's Special Initiative on Agribusiness/Cassava, (PSI) affected the RTIP implementation environment. An agreement was concluded between PSI and RTIP whereby RTIP was to supply improved planting material to cassava farmers providing inputs to PSI starch factories. It raised awareness about cassava and increased demand for RTIP planting materials to the benefit of RTIP. However, the considerable efforts that RTIP made to respect its PSI agreement detracted from efforts to realise other elements of its programme and PSI's different approach sometimes confused farmers.
The evaluation found that the rationale for RTIP was very relevant for rural poverty reduction in the Ghanaian context at the time that RTIP was designed, as were RTIP objectives. The components that made up RTIP were also relevant to achieving those objectives. However, the original omission of post-production and marketing activities from the original programme was a major flaw in its design. The component subsequently added to cover this area was inadequate, as was the priority afforded by the PCO. This has been the most important factor that has kept RTIP from reaching its overall objectives so far.
Looking at RTIP results, the planting materials multiplication system established an efficient three tier system for multiplication and distribution of four improved cassava varieties in 50 districts between 1999 and 2002. The recorded number of farmers who have accessed new materials is about 105 000. If those farmers that have accessed improved varieties but have not been recorded were to be included, the number would be considerably higher. For sweet potatoes, two improved varieties have been multiplied and distributed to a total of some 14 500 farmers. Attention to yam and cocoyam by RTIP has been very limited, with no significant achievements to date in the multiplication and distribution of improved varieties of these crops. This was largely due to the fact that there were no improved varieties available to multiply and distribute. Actions that were undertaken included promotion and multiplication of two highly valued local yam cultivars as well as a campaign for propagation of coco-yam late in the programme.
The component for adaptive research served as a vehicle for undertaking more than 60 research projects submitted by Ghanaian researchers in more than a dozen different government and academic institutions on agronomic areas as provided for in the RTIP appraisal report. During the course of implementation RTIP added investigation into integrated pest management, post production and marketing to address newly arising issues. Among this component's results are the release of five new cassava varieties, upcoming releases of sweet potato, yam, cocoyam and more cassava varieties currently in the pipeline, recommended practices for control of Imperata cylindrica, control of tuber rot and options for maintaining soil fertility. Overall, the activities have contributed significantly to the knowledge base in Ghana on root and tuber crops. However, the component limitations included its approach to determination of research priorities, inadequate consideration of socio-economic factors in the selection and evaluation of varieties and practices to be pursued for development, limited dissemination of research results, weak links with extension, and minimal integration with the workings of the PCO and other components.
In the IPM component results have been below target due to delays in construction of two of the three planned insectaries. However, predators furnished by the RTIP appear to have led to the successful control of the Green Mite Mononychelus tanajoa in seven of the ten regions of Ghana. Efforts to control Larger Grain Borer were made on a limited scale, with correspondingly limited results to date. Seventeen Farmer Field Schools, established in fifteen districts, served as vehicles for IPM and basic root crop cultivation techniques. The evaluation team considers the FFS to have been successful, but arguably inadequate in number and too resource intensive, reaching a total of only about 600-700 farmers in 20% of the 76 districts currently in RTIP. Further, FFSs had weak links with the adaptive research component resulting in missed opportunities to the detriment of both researchers and beneficiaries.
In the Community Support and Mobilisation component, RTIP has formed some 9 800 groups, about 96% of the groups were for production, 3% processing and less than 1% marketing. By and large groups were formed for specific temporary purposes. The evaluation team considers that groups formed have served as effective tools for the distribution of planting material and transmission of knowledge on improved cultivation practices. However, according to the BAS only 3% of respondents liked the group formation efforts and an estimated 80% of groups formed were either non-functional or disintegrated at the time of the study in July 2003. Processing and marketing groups, often based on pre-existing relationships among members, were more cohesive. Other efforts funded under this component included upgrading of government and NGO staff skills in outreach to beneficiaries, training of beneficiaries in improved crop utilisation, strengthening links to private sector processors, and an information, education and communication campaign. These were implemented roughly as anticipated, but suffered from a lack of a vision or overall strategy pulling together these disparate activities in a unified effort with a clearly defined purpose.
Under the post production and marketing component, RTIP collected information related to storage, processing and utilisation of roots and tubers, but stopped short of fully exploiting that information. Similarly, a study of marketing issues was undertaken. Improved cassava graters, stoves for roasting and screw presses were developed. Training modules were developed in cassava processing and in use of sweet potatoes. They were delivered to more than 1 500 people including MOFA staff, small-scale processors, bakers, and farmers. However, the PCO did not secure the staff and technical expertise to give this component the due importance. Cooperation between RTIP and the IFAD-funded Village Infrastructure Project in this field never materialised, despite being formalised in a Project Working Agreement in November 1998 as a condition of loan effectiveness. The lack of attention by MOFA and RTIP's PCO to the resolution of these problems is surprising given the seriousness of post harvest and marketing issues facing the farmers.
Results and performance in the Programme Management component were well above average, rising to the challenge of the design of this ambitious nationwide project. The PCO succeeded in creating working relationships with numerous different organisations and institutions throughout the country. It made substantial achievements in plant multiplication and distribution, integrated pest management and adaptive research. By and large, the PCO administration of the implementation of physical and financial activities, including disbursement, accounting and reporting, has also been good. The PCO showed weaknesses in two respects. Firstly, it tended to focus more on the technical and scientific sides of what was needed to achieve its overall goal, shying away from fully dedicating itself to some of the harder to solve, but essential issues on the economic and social side of the investment. Secondly, RTIP management appears to have allowed its energies to become fully absorbed in the logistics and practical details of achieving the physical and financial targets. Given the size and complexity of the task this is understandable. Nonetheless, it took insufficient time to consider to whether RTIP's implementation was leading to the desired longer term outcomes and impact.
RTIP, to its credit, has been reasonably effective in reaching the specific objectives of its individual components, even if there have been shortcomings and weaknesses in each component as alluded to in the descriptions of component results and performance above. It has concentrated most of its efforts on and been most effective in the planting material multiplication and distribution component, while it has been least effective in the added-on component for post-production and marketing.
It is still somewhat early to judge how effective RTIP in its entirety will be in achieving at completion its overall goals of enhanced food security and improved incomes of resource poor farmers. The RTIP log frame targets identified three indicators for achieving its goals: 720 000 beneficiaries reached; calorie consumption during lean season in beneficiary households increased by 20%; incomes in beneficiary households increased by 15%. RTIP records show it having reached 120 000 households to date. Nonetheless, the evaluation team considers that it may well reach close to 720 000 households by the end of the project. RTIP does not have records on income or calorie consumption that would allow similar assessments of whether it will be effective in reaching its targets in those areas.
Information collected by the evaluation team and drawn from the BAS indicate that there are very likely to have been income increases above 15% for the households of the more than 2 000 farmers who have participated in RTIP as secondary multipliers. In addition, the 14 500 sweet potato growers who adopted RTIP distributed varieties had output increases and faced ready markets are also very likely to have realized such income gains. However, it appears that the majority of ordinary farmers who adopted improved cassava varieties will have had little if any income increases.
Based on RTIP data the evaluation team considers that ordinary cassava farmers have achieved yield increases of up to 40%. For this reason, despite the absence of data on calorie consumption, the team considers that the food security of participating households has improved. Yet increased yields have not translated directly into increased incomes for ordinary farmers for several reasons. Firstly, revenue from cassava occupies a relatively limited share of total household revenue. Furthermore, there has been local inflation of 65% during the period January 2001 to May 2003, increased production costs associated with RTIP-introduced practices and varieties, relatively lower prices for the RTIP varieties, and a decline in cassava prices generally – most likely due to the overall increase in output levels in local markets as a result of RTIP and PSI3.
Notwithstanding the fact that RTIP's ultimate impact on the household income of resource poor cassava farmers remains to be seen, there are some indications that the investment will prove to have been relatively efficient in terms of its overall benefits to Ghana and the Ghanaian economy. Roughly speaking, the evaluation team estimates that there have been food security benefits for some 80 000 producer households (out of about 120 000 reached) and income benefits for some 16 000 households. Other, observable economy-wide benefits to-date from the RTIP investment of USD ten million, (about USD 14 per beneficiary household, based on the total number of beneficiaries targeted) include improved work processes and systems related to support of these crops by MOFA and other agricultural sector institutions. There have been lower consumer prices for root and tuber products. Further, RTIP has fostered a nascent potential for cassava exports, for low cost cassava supplies to agro-processing industries and for decreased dependency on imported wheat flour. The utilization of project facilities and services is high. On the whole the standards of RTIP services have been high despite some variation by component. Actual costs or expenditures have corresponded, for the most part, to appraisal estimates and there have been no significant implementation delays.
18. The evaluation assessed the rural poverty impact of RTIP in the six standard domains of impact set by IFAD: financial and physical assets, human assets, social capital, food security, environment and institutions and policies. The evaluation team rated RTIP overall impact on beneficiaries as modest.
So far the project has had its most widespread impact in the domain of food security. It is considered very likely that food security improvements will spread as RTIP reaches more households and that the changes in food security will be sustained given RTIP's impact on the farming technology and practices.
RTIP also made a positive impact in the domain of human assets in terms of access to information. The information, education and communication campaign of RTIP, including a national media campaign, training activities, and Farmer Field Schools have all contributed to achievement of this impact by making knowledge of the production, uses, processing and marketing of root and tuber crops more widespread among resource poor farmers. By way of doing this RTIP also, necessarily, deepened and widened available information on these crops among the scientific community, government staff and the public at large. In the same vein, RTIP had a positive, albeit limited, impact on the public sector institutions associated with its implementation and the services they provided.
As noted above, RTIP has not had the impact on household income levels on the scale that it aspired to. Nor has RTIP made a significant impact on social capital and empowerment of resource poor households that it might have had were more attention to have been given to these matters. The evaluation team found no evidence that RTIP has had any impact on policies or regulatory frameworks that affect the rural poor. Whereas, it will very likely have had a negative impact on the environment in so far as increased and intensified cassava production worsens soil nutrition status.
An estimated 39% of recorded RTIP beneficiaries were women, however RTIP efforts to ensure equitable impact, by gender, were limited. This was a weakness in both the design and the implementation of the project. Notwithstanding the fact that women were to be considered priority members of the RTIP target group, according to the Appraisal Report, the project did not explicitly monitor its impact on women, nor did it systematically disaggregate data by gender in its records or analyzes. RTIP did not adequately compensate for the missing collaboration with the IFAD-financed VIP project that was to involve women through its activities in processing of root and tuber crops. The project overlooked the opportunity it had to take into consideration gender specific needs and priorities in the implementation of RTIP research and extension components. Likewise, it neglected to actively promote the participation of women in the secondary and tertiary multiplication of improved planting materials.
The evaluation team has rated as "highly likely" the sustainability of RTIP impact in those areas where change was farthest reaching, i.e. food security (including farming technology, agricultural production and lower frequency of food shortages) and access to information. Whereas, it is considered unlikely that RTIP's modest impact on social capital and on services provided by public sector institutions will be sustained without further support.
The design of RTIP, nation-wide in scope with a focus on one commodity, was itself an innovation at a time when the lion's share of IFAD financing was going to area-based rural development projects, irrigation and rural finance. Further, the RTIP design responds to the identification of roots and tubers as "poor man's" commodities. The assumption that by supporting the development of these crops RTIP would benefit the poor people who grew them was innovative as an approach to poverty reduction. Yet it proved to be only partly true. The innovative choice to focus on a commodity was useful as far as it went, but it had limited impact on the levels of poverty of its producers in so far as it focussed primarily on production and almost exclusively on production of cassava. Insufficient attention was given to the characteristics of new varieties in the light of the role of the selected crop in the economy of a resource poor household.
As a nation-wide project RTIP cannot be up-scaled, however it does offer some useful lessons with respect to replicability. In particular, RTIP's three-tier planting materials multiplication and distribution system proved to be a successful one that merits consideration for replication in other crop improvement programmes. Furthermore, RTIP's development of an informal network whereby it exploited the knowledge and resources of a host of institutions and resource persons throughout the country and beyond provides a model that could and should be replicated by other projects and institutions. RTIP collaboration with IITA and the exchange of experiences with IFAD projects on roots and tuber crops in the region are also excellent features that merit replication.
The performance of the central partners in the basic tasks of project design and implementation, including loan administration, technical backstopping, and implementation support has been without any major problems. The main weakness in IFAD performance was in the design of the project with its insufficient or incomplete approach to targeting and its choice of an unsuccessful strategy for covering processing and marketing questions. To its credit, it has added intellectual and financial resources to complement the supervision process by the cooperating institution. The physical presence in country of the World Bank as the cooperating institution has been a strength in terms of accessibility by the project and resultant efficiency of loan administration. However, this has also had drawbacks where the Bank has become over-involved in budget details of individual project activities and expenditures. Its took an interesting and innovative "team" approach to the implementation support dimension of supervision that brought benefits to RTIP by exposing it to a wide range of knowledgeable experts. A drawback of this approach that would require correction was that responsibilities and accountability were sometimes too diffuse.
The International Institute for Tropical Agriculture (IITA) proved to be a very valuable partner in providing technical backstopping to RTIP through an IFAD technical assistance grant. Their involvement was a successful feature of project design and of project implementation. The fact that IITA has also supported other IFAD efforts in root and tuber crops in the region was an added benefit that has facilitated even further sharing of knowledge and exchange of experiences. The performance of RTIP's limited number of NGO's and CBO's partners was good and RTIP benefited from their outreach capacity and knowledge of local conditions.
The overall assessment of the evaluation team is that, given its ambitious size and its complexity, RTIP has been well-managed and successfully implemented with good results in most areas. Its greatest strength was in creating a well-functioning nationwide system for multiplication and distribution of improved planting materials. In the process it strengthened numerous institutions, increased farmer access to information, improved production practices and contributed to better household food security. By contrast it fell short of its full potential in terms of poverty reduction impact, due in part to deficiencies in the RTIP's design and in part to RTIP's incomplete or inadequate implementation of its planned activities.
The most serious design deficiency was what proved to be unrealistic and unsuccessful provisions for covering post-production and marketing issues. This design deficiency was compounded by inadequate attention to these same issues in implementation. Of comparable gravity was the deficiency in RTIP's design with respect to targeting. RTIP lacked provisions that could and should have been made to take a "pro-poor" approach in each of its components. It also lacked measures to ensure a better gender balance in the accrual of benefits. As a result, during RTIP implementation, management and technicians put technical considerations before target group considerations, meaning that impact on farmer incomes was not taken adequately to heart. During implementation RTIP also over-emphasized cassava with relative neglect of yam and cocoyam that meant neglect of the zones and households that favour those crops.
The recommendations for the remainder of the RTIP implementation period and the preparation of future investments are:
- Limitation of further distribution of improved cassava planting materials to areas with proven high demand for cassava for processing, i.e. PSI areas,
- Completion of IPM infrastructure and conducting a study to assess financial and economic impact of RTIP IPM approaches
- Continuation of varietal release programme, with production of extension materials and the development of a web-site to store and share information generated by RTIP
- Expansion of Adaptive Research component to include other research needed to achieve RTIP objectives
- Concentration of support to groups on processing groups and producer groups working with the Ayensu Starch Company
- Recruitment of 1-2 officers for post-production and marketing component to review existing situation and help identify needs in an eventual Phase II investment
- Review of the current M&E system to pare down collection and processing of data on RTIP physical and financial progress
The Office of Evaluation supports a second phase investment to follow up and consolidate investments and activities that have been undertaken through RTIP to-date. In this context its recommendations are the following:
- Future investments by the Government of Ghana in support of the development of particular crops, as in RTIP, should be designed to support the entire vertically integrated commodity chain.
- Future IFAD investments in the development of root and tuber crops in Ghana should maintain improved food security and increased household incomes for the rural poor as their overall goals.
- Past assumptions about crop sector development and its impact on poor rural households should be carefully re-examined in the design and in the implementation of future investments.
- Large scale campaigns for the multiplication and dissemination of improved varieties should be used to reduce rural poverty only once socio-economic benefits to poor farmers who adopt improved materials have been fully assessed and only in cases where it is clear that the market can absorb a large supply response, as with the government PSI initiative.
- Agricultural research and farmer field schools, or other extension activities, should be demand-led. Farmer priorities must be routinely ascertained and given ample weight alongside technical and agricultural policy considerations.
- A study should be undertaken of the comparative costs and benefits of farmer field schools versus those of normal extension practices at the district level before further investments are made in this approach.
- Future investments for reducing poverty through investment in root and tuber crops should emphasise activities related to post-harvest, marketing and development of new market opportunities.
- Among post-harvest and marketing activities that should be considered for support are: (i) appraisal of technical and financial viability of existing processing equipment; (ii) training and advisory services on processing techniques; (iii) training on hygiene, health and environmental issues at processing sites; (iv) advisory services on packaging and labelling; (v) improved storage methods, (vi) regular dissemination by radio of price information; (ix) promotion of linkages between producers, processors, and traders on outputs and equipment and, (vii) elaboration of various financing models and arrangements with financial institutions to fund processing equipment and working capital requirements.
Smallholder Livestock Rehabilitation Project (2004)
Completion evaluation 1
Project design. The Smallholder Livestock Rehabilitation Project (SLRP) was the first assistance provided by IFAD to Lebanon . The project, designed as a post-conflict intervention, started at a time when the country was emerging from 17 years of civil war and closed at the end of June 2002. The Completion Evaluation of the project had as objectives assessing project achievements and impact and draw lessons from its experience for future similar IFAD interventions in the Near East Region. In undertaking the evaluation the field mission fully implemented the Methodological Framework for Project Evaluation developed by the IFAD Office of Evaluation (OE) in 2002 and followed its rating system.2
At the time of project design, basic services and infrastructure in the project area needed rehabilitation, and the Ministry of Agriculture (MOA), which was gradually being re-organized, had limited experience in the implementation of internationally funded projects. The project was designed with the ultimate aim of addressing the needs of the poor and disadvantaged in the post-conflict reconstruction of the project area. The following main objectives were conceived: (i) provide the stimulus to re-establish smallholders livestock production in the Bekaa Valley ; (ii) replenish some of smallholders lost stock in the project area; (iii) provide the basis for sustainable development in the livestock sector and increase the income of the poor farm families and women who derive most of their income from livestock; and (iv) develop agricultural support services. The last objective was to be achieved initially by providing institutional strengthening for the MOA, both to enable the Ministry to implement SLRP, and also to enhance its overall capacity as an agent for agricultural rehabilitation and development. At the time, IFAD thinking on post-conflict support had not been articulated, hence there were no clear policies or guidelines for the designers to follow to achieve these goals.
These objectives were translated into four components in the design: (i) Strengthening Support Services (26% of base costs); (ii) Agricultural Credit (49%); (iii) Off-farm Income Generating Activities (IGAs) and studies – including a programme for Rural Women (RW) (12%); and (iv) support for the Project Management Unit (PMU) (13%). The target groups proposed were those with small landholdings, less than 4 ha, and with household earnings of no more than about 40% of the average per capita income (about 8500 households). The total project cost, including contingencies, was estimated at USD 21.9 million, of which IFAD provided a loan of about USD 10 million on intermediate terms, and OPEC cofinanced with a loan of nearly USD 5 million. The Cooperating Institution for the project was the Arab Fund for Economic and Social Development (AFESD).
During the period of project implementation the economy made great strides, so that by 1998 the national average GNP per capita had returned to USD 3 500, and inflation, which had been 100% in 1992, had dropped to just 5% by 1999. However, the proportion of the government budget allocated to agriculture had consistently fallen, so that by 2002 the MOA's budget accounted for just 0.6% of all government spending. GRL policies for agriculture over this period have generally emphasised the role of the private sector for investment, and sought to reduce the production costs of smallholder agriculture; the high cost structure was considered to be the result of the very large number of small "traditional" holdings. The open nature of the economy has meant that local producers have consistently had to compete with cheaper imports and small-scale producers of milk and meat have faced difficult times, characterised by falling prices. Presently, about 75% of red meat is imported, whilst Lebanon produces about half of its consumption of dairy products.
Design changes . Whilst the implementing agency for the project has been the MOA, a PMU was established in the Bekaa: this required an act of Parliament and gave the PMU considerable autonomy. A number of changes were introduced to project design by the PMU, with approval from IFAD and the CI, based on studies undertaken during implementation.
A milk marketing study (recommended by the design) was conducted in 1995. Its main conclusion was to propose the establishment of milk collection centres (MCCs), in order to test and consolidate the outputs from small producers and to increase their market leverage. Four MCCs had been constructed in the Bekaa at the time of the evaluation, and a further eight in total have either been constructed or commissioned elsewhere. Capital required for these MCCs was mostly obtained by re-allocations from the small ruminant programme that has been reduced in size. This design change had implications for the direction of the project, and re-oriented it much more towards dairying as its main activity, with an emphasis on milk marketing. The lack of development of the small ruminants' programme, significantly affected the achievement of one of the main project objectives regarding the increase in income of poor farm families.
As an extension of the fodder development programme, in 2003 the project established a sophisticated feed testing laboratory, also financing this from reallocations from the sheep and goat credit programme. This facility was thought necessary because of farmer uncertainty over the quality of locally produced feed. In addition, cooperatives for rural women were identified as the preferred means of group formation, and project funds were re-directed to provide loans for these cooperatives as well.
None of the above activities were included in the original design and both IFAD and the CI demonstrated flexibility in accommodating the requested changes. Although the changes in total were significant, their overall impact on the coherence between project objectives and implemented activities has never been addressed. This was partly because an effective monitoring and information system, a participatory M&E mechanism, and regular beneficiary feedback were not established, and partly because the Mid-term Review (MTR) recommended by the project design was not undertaken.
Summary implementation results 3
Restocking. The project has made available a total of 1 430 high quality imported pregnant heifers for distribution. The first consignment of haifers arrived three years after project effectiveness. By the end of the first year under beneficiary ownership, a 26% had died (compared with 3% assumed at appraisal), leaving just 1 012 surviving animals (50% of net target4). Livestock insurance was a required feature of the loan arrangements for farmers, and was used to compensate them for these cattle losses. The final importation of 600 animals, to reach the project target of 2 000, was cancelled because of disease problems in the supplying country and price escalation.
Project data do not indicate the actual number of households that received heifers, and the evaluation found it difficult to confirm the status of the beneficiaries, either from the data or from field visits. The design called for two heifers to be allocated per household, but this was abandoned at an early stage, and heifers were distributed individually. Many heifers were apparently quickly on-passed, so "end-user" households mostly received three or more animals. The livestock activities included also a small ruminant fattening programme: this was operational for two years and achieved 80% of appraisal estimates.
Support services. The project introduced a range of support services for livestock. Artificial Insemination (AI) was introduced to the Bekaa and this gave the possibility of much higher performance for dairy farms throughout the valley compared to local bulls, enhancing milk production capability from about 3 500 litres to 6 000 litres plus. The programme also increased farmer awareness of this greater potential. All (dairy) cattle were vaccinated against Foot and Mouth Disease in most years, and to a lesser extent, against some other diseases: considerable numbers of productive sheep and goats were vaccinated against key diseases. This was most useful, laid the foundation for a better animal health status, and made farmers appreciate the value of vaccinations. The project's Extension Officers (EOs) have introduced more modern animal husbandry and milk production technology to more than 600 producers, resulting in their ability to produce milk of much higher quality than before.
The livestock activities included a small fodder promotion programme which introduced crop/legume mixtures, conducted field days and produced and distributed a significant quantity of improved fodder seeds. The feed analysis laboratory was constructed as part of this programme, but its impact has been limited, as has been the effect of interventions to improve rangeland stability and conservation. A Research programme into the upgrading of sheep and goats was begun during the project and led to the progressive build-up of flocks at the Lebanese Agricultural Research Institute (LARI). However, most of the animals were disposed of in 2002 when SLRP stopped funding the activities at LARI and passed over the control and management of project-sponsored activities and resources.
Milk marketing. The first Milk Collection Centre (MCC) was constructed at Bar Elias in 1998, and the project contracted an existing milk collection cooperative to arrange supplies. Three more MCCs were commissioned in 2001 and six more cooperatives were contracted to supply them with milk, although none of these cooperatives had ever collected milk before. Technically the MCCs have performed well, producing milk of improved quality. However, it was assumed that the processors would pay a higher price based on quality and that the Government of the Republic of Lebanon (GRL) would introduce and enforce a recommended support price – this has not happened. Analysis from Bar Elias indicates that MCC operations add about 28% to the farm gate milk price. The premium processors' willingness to pay for higher quality supplies has resulted in an increase in the farm gate price of milk compared to the price paid by traditional collectors: this has been of benefit to small producers, but is steadily reducing. During the period April 2002 – May 2003, all but the first milk collection cooperative withdrew their services, so that only one MCC was operational during the evaluation. The evaluation found that the cooperatives had often not paid their producers on time, in spite of borrowing start-up and operating capital. This does not augur well for the future of these activities and their sustainability.
Credit. Lebanon has long been regarded as one of the region's most important financial centres. Private banks in Lebanon have enjoyed government support and attracted deposits from both national and regional sources. Today Lebanon 's financial sector boasts 72 banks with a wide network of over 700 branches, of which 194 are located in the Bekaa. However, by and large, Lebanon 's financial institutions are essentially urban.
Following a credit study undertaken in early implementation the project opted to take direct responsibility for managing the credit line and two credit units were established in the PMU, one for agricultural credit and the other for rural women, covering the loans for IGAs. The credit units developed the criteria and procedures for the selection of borrowers and the terms of the loans following the base line survey and specified the socio-economic characteristics of the credit target group. A commercial bank has served as the financial intermediary for the project, but did not risk any of its own funds. The bank has also been the depository for the credit funds. The project calculated standard loan amounts for the different (livestock) activities to be financed; generally the beneficiaries were required to provide a 25% deposit and offer collateral and two guarantors. Livestock loan sizes varied from USD 4 600 for a dairy unit, to USD 2 270 for lamb fattening. Importantly, loans provided to women were not linked to a specific type of economic activity.
In just under 10 years of operations the project has managed to distribute about 55% of its credit funds: this is estimated following the cancellation of credit for the small ruminant programme and the introduction of new loan categories for cooperatives. Just over USD 3.5 million has been disbursed and 3 272 loans have been opened (but not all are continuing, the net figure is 2 683): this distribution performance is considered very creditable. However, there has been a marked contrast in the project's experience with loan repayments. In the case of livestock loans, repayments have been exceptionally low, whilst in the case of the women's programme, loan repayments have been exemplary. Summary repayment rates to date are:
- Dairy Cows: (i) Dutch source- 18.3%; (ii) French source – 9.3%
- Fattening lambs/sheep: 12.5%
- Loans to milk collection cooperatives: 14%
- Individual loans for Rural Women's: 84%
- Loans to rural women's cooperatives: 67.7%
According to project staff, the most important factor in low repayments for livestock loans has been the common perception among loan recipients that loans are de-facto 'disguised grants'. Further complicating the issue is the fact that some of the farmers considered that the loans were compensation from the Government for destruction of their hashish (marijuana) crops. An underlying assumption of the credit design was that the target group, given their meagre resources, could not meet the credit criteria required by commercial banks. That is, most project borrowers had never had loans before. Managing a first time loan that could be, as in the case of dairying, quite large (sometimes higher than annual incomes) was not an easy undertaking for many borrowers.
Women's programme. All 500 of the loans for rural women were distributed between 1996 and 1998. Overall, the Rural Women's Unit in the PMU estimated that about 70% of the beneficiaries had been poor rural women: in some instances, it is possible that the loan conditions may have been prohibitive to poorer women and excluded them from participating in the project. The loans were used mostly for food processing. Several factors are thought to explain the high repayment rates: activities were of interest to the beneficiaries; adequate pre-training was provided; a supportive environment was built based on trust between the beneficiaries and the extension agents; and the continuous monitoring of beneficiaries' activities by the extension agents. Training organized by the Rural Women's Unit (RWU) on matters related to loan management has been extensive and, in essence, women who took loans were 'adopted' by the project's RWU. Three years after exposure to USD 500 loans, women were ready for larger credit opportunities, provided through the medium of cooperatives.
A further contributory reason was the healthy profits made from most activities. As a result of high repayment rates the Rural Women Unit (RWU) was able to establish a revolving fund, thereby disbursing an additional 459 loans. Overall, therefore, 959 women (180% of target) had benefited from the IGA credit activities at the time of the evaluation. The women's credit programme is considered by the evaluation to be very successful. The main problem reported to the evaluation was recent difficulties with marketing.
Project design called for a group formation mechanism to be introduced as part of the women's programme. Between 1999 and 2001, 20 rural women's food production cooperatives were registered: they were the first of their kind in the Bekaa and perhaps also in Lebanon . The cooperatives were also permitted to apply for loans from the project in their own right, this investment is in addition to the loans granted to individual women, which are usually "pooled" in the cooperative and invested jointly.
An important impact of the cooperatives has been that they provided employment for their members. Members can earn between USD 150 - 200 dollars per month, for five to eight months a year. In addition to direct employment, members also receive a share of the cooperatives' profits, but often opt to leave these dividends with the cooperative. In 2002, nine cooperatives joined together to form an Association, to improve their marketing possibilities, this is considered a promising development.
Project management. The decree granting the PMU autonomy allowed the PMU to employ its own staff, prepare independent budgets, set procurement conditions and devise lending arrangements. Staff numbers have changed over time: in the earlier years (to 1998) a lot of staff were seconded from MOA and LARI; subsequently these numbers have fallen (with the end of animal distribution activities and the hand over of the breeding programme etc to LARI), to be more than replaced by direct employment of staff for the MCCs. The present staff strength is 114, of which only nine are now seconded (c.f. 44 in 1999). The evaluation was impressed with the standards of staffing throughout the project; although it is difficult to relate present standards to the early 1990s, there appears to be now no lack of technical competence. Project budgets have been prepared annually and include counterpart funds, which have always been provided by MOA, although with some delays. The project now requires approximately LBP 4 billion for annual expenditures (USD 2.7 million), which is nearly three times the annual operational cost estimated by the Appraisal; this accounts for 11% of the MOA budget (in 2003).
Institutional strengthening . Interviews with staff confirmed that the Technical Assistance (TA) and training provided had been both meaningful for their project tasks and matched their needs. The evaluation found that the recipients of this training were either still working for the project or the Ministry, or were in a closely aligned occupation. It was concluded that the training and TA provided were both necessary for the start of the project and are having an ongoing impact in increasing the capacity of the Ministry, as many of the seconded staff have now returned to their original roles.
Monitoring and Evaluation (M&E). Despite the emphasis in the design on institutional strengthening, the TA for M&E was never recruited – this is the only example of TA not being used as designed in the project. The baseline survey was undertaken in 1994 and provided useful benchmark data for the selection of project beneficiaries, however, no key indicators were ever derived from the survey. Possibly as a result, whilst the monitoring has been adequate the project has had no impact indicators to guide management decisions. For example, it is not possible to trace the location of animals supplied through SLRP or to collect performance data which might provide proxy indicators of impact.
Relevance. In terms of replenishing the livestock resources of the Bekaa the evaluation found the original project objectives to have been soundly based at design. However, the slow start to the project and the late arrival and distribution of animals reduced the relevance of the dairy intervention for the intended target group (see also paragraph 46). The project objective of restoring or establishing livestock support services was very valid at design and remains as valid at the time of evaluation, either through government services or through the development of private sector operators. The objective of establishing an institutionalised credit mechanism for small and poor rural households remains relevant. Institutional strengthening for the MOA was identified in the design as a vital requirement for implementation, on the basis that services had been badly disrupted and the civil war had stopped almost all programmes of Human Resource Development (HRD). Judged from staff assessments and performance the objective was relevant as designed, but is now no longer necessary.
The evaluation concluded that, while project objectives were relevant within the Government's development ambitions at the time of design, and many remained substantially relevant at the time of evaluation, the relevance of some of these objectives for rural poverty reduction in the Bekaa has now been reduced. This is due, in part, to the nature of interventions for institutional reconstruction in post conflict situations and in part due to del ays in implementing some activities.
Effectiveness . Effectiveness is defined as the extent to which the objectives have been achieved, and the extent to which achievements have impacted on the lives of the rural poor as intended at design. The evaluation concluded that for rural women beneficiaries the project had been particularly effective in enhancing their livelihoods, directly by increasing access to credit, improving incomes and upgrading skill and indirectly in terms of gender equity and empowerment. It has fostered the development of institutions for the rural poor through the formation of cooperatives for rural women, which were accessible to men as well. The project has also been very effective in improving access to information and knowledge for all project beneficiaries through its training programme, and the objective of institutional strengthening for the MOA has largely been achieved.
Although the project will have an increasing impact on the dairy industry in the Bekaa, through organic growth in the dairy herd and its followers, uncertainty remains as to actual project reach, the socio-economic status of the beneficiaries, and the extent to which project benefits have and will reach the IFAD target groups. This, and the cancellation of the small ruminant activities, originally targeted to smallholders, have meant that the restocking objectives, as formulated at design, have not been achieved to a significant extent. On balance, therefore, effectiveness of the project has been substantial but could have been higher with respect to poverty reduction objectives.
Efficiency. The evaluation was unable to prepare an overall ex-post Economic Rate of Return (ERR) for the project, because of the lack of recording of benefits and beneficiary numbers. Given these constraints, the evaluation prepared farm models for the dairy enterprises, which absorbed the bulk of the project resources. Overall, it was concluded that the incremental benefits for the dairy industry as a whole in the Bekaa from the introduction of the dairy herd were likely to be high, compared to the costs incurred by the project in providing these benefits. Despite this overall conclusion, the farm models suggest that the introduced technology was not appropriate for many of the small scale farmers who were the intended target group of the project, rather it was appropriate only for those farmers able to sell their milk at high local prices in niche markets, or to large scale, high productivity farmers. Project efficiency, was therefore not high compared to original design expectations. Overall, project performance has been substantial.
Impact on physical and financial assets. For successful dairy farmers physical and financial assets have increased, sometimes quite substantially. However, this does not apply to the smaller or poorer farmers, who generally considered that dairying was high risk and did not reduce vulnerability. The evaluation found that lamb fattening was no longer considered profitable and had not contributed to increasing household physical or financial assets. For most of the rural women who had taken loans, financial assets had increased. This conclusion arises from the high levels of profit achieved in many IGAs, but is also influenced by the possibility of employment in the cooperatives.
Impact on human assets. The project has provided a significant amount of training for all beneficiaries. The evaluation concluded that this training had a useful effect in terms of increasing levels of knowledge and skills and that these were being applied in improving living standards. However, the project has also resulted in increased workloads for many households. Women's workloads have increased through their employment in the cooperatives, although only in the processing season. Evidence also suggests that men's workloads, and probably children's workloads as well, have increased with caring for more livestock.
Impact on social capital and empowerment. There have been two diverse experiences in social capital in SLRP. Dairy farmers did not experience any empowerment, rather they realised their weakness in the face of milk collection cooperatives which they were unable to influence. In contrast, through the Rural Women Programme (RWP), over a thousand women have become members of women's cooperatives; cooperative working has allowed members to put aside their traditional affiliations, uniting them in facing common goals and challenges. The social cohesion established is considered sustainable and could have profound local impacts. The women component, however, absorbed only a small fraction of total project expenditure.
Impact on food security. The evaluation concluded that the project had moderately increased household food security in the project area. For dairy farmers some of the extra milk produced has been kept for home consumption, whilst women activities increased income (hence food security) directly through IGAs and indirectly through women's employment in cooperatives and access to products at wholesale prices. The reduced size of the small ruminants activities has probably resulted in forgone income and food security for poorer farm households. An unintended benefit for poor households, which are not dairy farmers, is that the drop in the price of milk may have made dairy products more affordable.
Impact on the environment and the communal resource base. The livestock activities have probably had a small positive impact on the environment, as most livestock are not grazed, but remain housed on the farms. The fodder produced to support them, if grown on-farm, is fertilised with farmyard manure, hence soil fertility and soil structure on these farms has probably improved as a result. The project introduced rangeland management and reseeding practices in the sheep grazing areas, with some positive effects.
Impact on institutions, policies and the regulatory framework. Whilst the project has improved access to credit for poor rural women, this was not associated with any changes to the existing institutional arrangements for the provision of financial services to the poor. The successful experience of the formation of the first women's cooperatives in the Bekaa has demonstrated the usefulness of cooperatives as self help organisations though it still remains of limited scale. The Institutional Strengthening provided for the MOA has enhanced the capabilities of some of the Ministry staff. The evaluation could find no impacts on the policy or regulatory frameworks.
Gender. Women's social status has effectively been enhanced as a result of the success of the RW cooperatives. More women are now able to work outside their homes, they became active members of cooperatives, have demonstrated their self-reliance, and as income earners, their decision-making role in the household has been strengthened.
Innovation. The project had some contextual innovative aspects. The model of women's self help cooperatives developed by the RWU was a very innovative approach for the Bekaa, and is probably replicable in other parts of Lebanon . There is some evidence that the YMCA may have already replicated this model elsewhere. The project introduced AI to the Bekaaa, which can also be regarded as a contextual innovation to improve genetic potential. Potential for replication and scaling up exists particularly if it is provided by competent and adequately motivated operators. This service can also be sustained through the private sector. The evaluation found, however, that AI could not substitute for bulls in the more remote rural areas, as intended.
Livestock. The circumstances of small farm dairy production have changed since appraisal and small-scale dairy enterprises may now be marginal or unprofitable at the current cost of concentrates and the level of milk prices. The livestock support services which can be offered through the private sector are likely to prove sustainable providing they are not undercut by competition with government and prices are affordable to small farmers. However, the sustainability of the services which are normally funded by government - extension and research – appear much less certain. In the case of animal health and livestock extension, useful capacity has been created in the MOA, but the extent to which it will be used will depend on the policies adopted for support for the livestock industry in general and the funding available. The MCCs and the milk collection cooperatives have had some positive impacts on milk prices, but it is questionable as to whether this can be sustained, in view of the withdrawal of most of the collection cooperatives and the continuing uncertainty over the operation of the MCCs themselves.
Credit. The project has made no provision for an autonomous self-reliant mechanism that can perpetuate the provision of credit (and other rural financial services) to the poor in the Bekaa. The way in which credit has been provided by SLRP has meant that at the end of project activities, credit would come to an end. In order to assess the sustainability of income generated, evidence is needed of the growth of the initial investment and/or reinvestment of profits. Evidence for this is scanty for livestock loans, but profits made by cooperatives often appear to be reinvested. Overall, women's loans for food processing, which are relatively flexible in that products can be adjusted to market requirements, are likely to be sustainable.
The Rural Women's Programme . The entrepreneurial and social development benefits which individual rural women have received through participating in the RWP have been considerable, and these are presently sustainable, without further inputs from the project. However, because the cooperatives are all relatively new, they are not in a position to "graduate" as fully independent entities, as yet. To ensure sustainability will require further support for the near future, especially with marketing. To date, the cooperative spirit and determination demonstrated has been remarkable, and augurs well for the future, if resources continue to be invested in this area in the near future. For the project as a whole, the likelihood of sustainability of its impact on rural poverty does not appear high.
IFAD and the Cooperating Institution (CI). Overall, the evaluation concluded that IFAD had performed reasonably well in facilitating a design that was appropriate to the circumstances at the time, but had performed modestly in providing guidance to the project in making strategic decisions during implementation. This is obviously shared with the CI. Project staff reported favourably on support received from the CI, whose reports have mostly indicated a very satisfactory implementation situation. However, the evaluation concluded that this was overly optimistic with respect to assessments of development impacts and sustainability, expected benefits and beneficiary participation. As a result, appropriate incentives were not created for the project to pursue systematically its original poverty reduction objectives. It is unclear how the CI assessed these development indicators, as no data were available. In addition, the evaluation would also question the very positive rating given to the M&E system given the limited beneficiaries and impact assessment during implementation.
Co-financiers . The OPEC fund co-financed the livestock credit activities; project sources reported that OPEC has been a full and active partner in the project, despite the problems with credit recovery.
Government and its Agencies (including project management). MOA facilitated the granting of autonomy to the PMU and has supported the project with seconded staff and counterpart funds. However, the slow start of the project and delays in procuring the animals reduced the relevance of the intervention for smallholders in the post-conflict situation. Although research programmes were started at LARI, and considerable investment in reconstruction took place, the programmes were only continued as long as SLRP provided the funding. As soon as they became the responsibility of LARI, they were stopped. For most of the project life the project management has effectively pursued the implementation in a dynamic and technically competent fashion, and has successfully implemented the training activities for staff and beneficiaries and the programme for rural women. However, the evaluation was concerned at some of the decisions reached during implementation, particularly those concerning aspects of project design, which have affected the direction of the project, its intended objectives, the extent of outreach to the target groups and the impacts achieved for poorer beneficiaries. Overall, partners' performance could have been higher in terms of influencing project implementation towards the achievement of its intended target group reach and poverty impact.
(i) - Creating Social Cohesion. An unexpected but very significant effect of the project has been the extent of social cohesion which has been fostered as a result of the formation of cooperatives for rural women. In Lebanon there are strong traditional allegiances that have been strengthened during the seventeen years of civil war. However, the cooperatives, formed as a means of bringing women together for their financial advantage, have led to the identification of common concerns and interests, thus diffusing barriers and building social capital to a large extent not aligned to established socio-economic and traditional structures. The impacts of this experience have been felt economically, and socially. Economically, the initial vulnerability and insecurity for rural women have largely been substituted by feelings of empowerment and equality, as incomes have increased and been shown to be sustainable. Socially, the collective work ethic, equal partnership among women and joint risk taking have drawn the group members together, facilitating the exchange of old allegiances with new. In addition, the formation of these cooperatives has given the Government a means of realising its policies for social and economic development in poor rural villages.
The opportunity exists to build on and extend this experience. The evaluation strongly recommends that a wide range of support for the present rural women's cooperatives be continued, as they are still new experiences and require assistance to ensure they can "graduate" to being fully independent. This experience should also be furth er studied and made use of in other IFAD supported interventions in the region.
(ii) - Restocking. The basic rationale for re-stocking is to restore lost production and provide the means for poor farmers to earn a living. In the case of SLRP, although the project was designed in 1991, the first consignment of heifers did not arrive until late in 1996 (three years after effectiveness and more than five years after project design). Five years was too long to wait. Inevitably, many livestock farmers would by then either have moved on, or found other means of replacing their lost stock. By 1996 restocking was possibly not what was required anyway, as it interfered with the commercial development then taking place in the dairy industry of the Bekaa. In addition, the heifers were only available as part of a credit package which, for the intended beneficiaries at project design, was far too large in comparison to their incomes, hence discouraging them to acquire it.
Whilst these arrangements may be appropriate in some development circumstances, they were hardly conducive to the prevailing post conflict situation in Lebanon . The requirement was for very speedy action to restore production, followed at a later date by considerations of re-couping some of the costs involved. For example the two-heifer units could have given to smaller and poorer farmers on the basis that the first two female calves were returned.
(iii) The Targeting of Dairy Interventions. In SLRP potential recipients of dairy cows were apparently selected on the basis of applications submitted by individuals who met the poverty criteria suggested in the Appraisal, augmented by findings from the baseline survey. Another consideration for the project was the need to ensure a reasonably balanced distribution throughout the project area. With hindsight , this approach created problems both with the support of the animals and also with marketing. These difficulties have reduced the poverty impacts of the intervention, and probably contributed to re-distribution of the animals to non-intended beneficiaries. For high producing dairy cattle to be the means of increasing incomes for poorer households, local markets for dairy products should have been readily available and, most importantly, ready access either to bulls or AI. Small dairy units (of two cows) located over a wide geographical area made meeting these needs exceptionally difficult. A preferred approach may have been first to develop local farmer organisations (such as cooperatives) based on a village or other defined areas, then to supply the animals. This would aim to increase the "density " of production so that, for example, collective milk marketing and the keeping of a bull might be justified as bringing benefits to all the producers.
(iv) Milk Collection Cooperatives. The project design suggested a possible role for cooperatives in milk collection, and nearly USD 1.5 million was allowed to develop this possibility. At the time of the evaluation most of the milk collection cooperatives had withdrawn their services, with the result that three of the four milk collection centres were shut. However, this should not be interpreted as meaning that cooperatives could never undertake this role. In fact the project experience suggests that milk collection cooperatives can be effective under the right circumstances. In a more general sense, it is important not to assume that all cooperatives are the same. The success of the RW cooperatives illustrates just how useful and beneficial properly constituted and run cooperatives can be in the development effort, whilst the modest experience with many of the milk collection cooperatives demonstrates some of the difficulties. This illustrates that where cooperatives are to be utilised in future IFAD-supported projects, great care needs to be exercised in their selection, effective management, and equity in the distribution of benefits to their members.
(v) Credit Mechanisms . The design of the credit interventions in SLRP was in accord with the credit approaches of the early 1990s, i.e. was concerned with 'opportunities provided', rather than as an 'approach to self-development' that needed to be gradually introduced and nurtured. The shortcomings of this approach in terms of development impacts and, particularly, sustainability have emerged from the implementation experience.
(vi) Size and Purpose of Loans. In addition, the credit experience also highlighted the importance of flexibility when it comes to how loans were to be utilised. Loans provided to women were not linked to a specific type of economic activity and three years after exposure to $500 loans, women were ready for larger credit opportunities, provided through the medium of cooperatives. By comparison, large livestock loans were provided for very specific purposes and were quite inflexible. For livestock, it was assumed that members of the target group, who may never have taken loans before, could successfully manage relatively larger credit linked to specific economic activities, whilst women, it was assumed, would be better served by smaller loans that could be used to finance various types of IGAs. The latter approach has proved more effective as well as being more consistent with reaching the intended target groups.
(vii) A Role for the Private Sector. With limited funding available through the MOA, an important element in the development of the livestock sector was the encouragement of the private operators to take an increasing role. This is an important plank of GRL policy. Although the delivery of AI services stopped when the project ended, it has been adopted since and is being continued by veterinarians and technicians working in the private sector. Vaccination and animal health services are similarly available, but in both cases have to compete with subsidised government services. If the government wishes toencourage private sector activities in the provision of AI, then it needs to limit its role to regulating the service and ensuring that technicians are properly trained and work to industry standards. For vaccinations, Government should regulate and control vaccine quality and type, and identify and confine its services to supplying the public-good vaccinations.
1/ The Completion Evaluation Mission was composed of Mr Frank Butcher , Mission Leader and Institutions Specialist; Dr Mouna Hashem, Sociologist and Gender Specialist; Mr Roderick Kennard, Livestock Expert; Mr Omar Imady, Credit Specialist; Ms Lea Joensen, Economist, IFAD Associate Professional Officer. Dr Mona Bishay , Deputy Director, IFAD’s Office of Evaluation, supervised the evaluation process throughout.
2/ Methodological Framework for Project Evaluation (IFAD’s document EC 2003/34//W.P.3, see also IFAD website and Appendix 3).
4/ The evaluation methodology followed uses three composite evaluation criteria: project performance (composed of relevance, effectiveness and efficiency) rural poverty impact (composed of six impact domains, sustainability, innovations and gender equality) and performance of partners (including IFAD, implementing agencies, and the cooperating institution). The ratings used to assess performance using these criteria are high (4), substantial (3), modest (2) and negligible (1). For the sustainability criterion the rating used is highly likely (4), likely (3), unlikely (2) and highly unlikely (1).
Scaling Up Poverty Reduction: A Global Learning Process, and Conference
Empowering the poor by shifting from a supply to a demand-driven approach
The Management of Natural Resources in the Southern Highlands Project (MARENASS) and the Development of the Puno-Cusco Corridor Project (CORREDOR) were launched by the Government of Peru in 1995 and 1997, respectively. Both projects operate in the Southern Highlands, where about 73 per cent of the rural population is poor, and 27 per cent is extremely poor. The projects work to reduce poverty by enabling families and communities to offer competitive goods and services that help free them from dependence on agriculture and herding.
Consulting with rural poor people during the planning phase is key. Both projects build on the earlier successes of the Promotion of Technology Transfer Project to Peasant Communities in the Highlands Project (FEAS), which ended in 1999. Funds are provided directly to rural poor people, who can contract technical assistance services of their choice. The shift away from a supply-driven system of extension to one of demand-driven assistance creates a more dynamic market.
The new USD 21.7 million Market Strengthening and Livelihood Diversification in the Southern Highlands Project will further build on IFAD's experience in Peru.
The four projects represent an investment of more than USD 98.6 million in the Peruvian Highlands. To date, the projects have reached about 1,000 communities and 1,100 farmer organizations – almost 100,000 families.
- Transferring decision-making power and control over project resources to rural communities helps increase incomes, assets and self-esteem.
- Trained local service providers are better able to respond to rural communities' needs.
- Relationships between local service providers and the community can bridge rural and urban areas and expand marketing and other small-business opportunities.
- Involving rural poor people in all stages of project planning and implementation legitimizes their perceptions and needs, creates realistic expectations and reduces controversy.
- "Innovative technology" competitions between communities are an effective way to encourage better resource management and sharing of ideas.
- Projects that are designed to be flexible enable practices to be abandoned or reinforced as necessary.
Communities that are directly responsible for managing their financial, natural and social resources have a sense of ownership over development activities. This in turn boosts self-esteem and empowers them to negotiate with local government, civil-society and other institutions for vital services and assistance. The impact has included:
- 500 farmer organizations have contracted their own technical assistance services for a total cost of USD 7.5 million, contributing USD 1 million themselves (FEAS).
- Almost a third of the families have set up new microenterprises (MARENASS).
- Incomes of rural poor families rose 64 per cent between 2000 and 2003 (MARENASS).
- 60 per cent of communities improved their household assets, such as homes, gardens and stables (MARENASS).
- About 2 300 microentrepreneurs and 1 800 artisans are engaged in new income-generating activities (CORREDOR).
- Women in 360 communities are setting up microenterprises with the support of a special fund administered by local women's groups.
- Every dollar spent by projects is getting a direct return of USD 3-5 in communities through the increased value of participants' assets.
The basic principles of the projects' demand-driven approach have been incorporated by the National Compensation and Social Development Fund, which is one of Peru's largest national development programmes. Some of the innovative features are being taken up by Peru's Ministry of Transport and the World Bank in a new project in the southern Sierra. These features are also being incorporated into European Union-financed projects in Guatemala and Chile. In addition, the World Bank has integrated the concept of transferring resources directly to beneficiaries into its rural development operations.
Community Development Project for the Rio Gavião Region (PROGAVIÃO) (2003)
This document contains an executive summary1 of the interim evaluation of the Community Development Project for the Rio Gavião Region (PROGAVIÃO). The evaluation – conducted in the state of Bahia , Brazil , in May-September 2003 – focused on analysing the sustainability of the rural development process launched by PROGAVIÃO. The possible execution of a second phase of PROGAVIÃO should be examined in light of these findings.
The PROGAVIÃO project was approved by IFAD's Executive Board in December 1995. In May 2003 the project was completing its sixth year of implementation since start-up in February 1997. The total project cost as estimated in the design was roughly USD 40.4 million, of which IFAD would finance approximately USD 20.1 million through a loan, the Government would provide USD 19.9 million and the beneficiaries would provide USD 0.4 million. The cooperating institution is the United Nations Office for Project Services (UNOPS).
The implementing agency is the Regional Action and Development Agency (CAR) of the Planning, Science and Technology Secretariat of the State of Bahia (SEPLANTEC).
Main design features and implementation outcomes
The Rio Gavião region is situated in the southern central part of the Brazilian state of Bahia , on the edge of the so-called "Drought Polygon" of North-East Brazil . It is characterized by very harsh environmental conditions and a historical and socio-economic setting that has led to extensive rural poverty. The project area covers a surface of 11 718 km² and is spread out over 13 municipalities (municípios) . Of the total population of 40 000 families, 70% live in rural areas.
The project's formulation observed the following strategic thrusts: (a) strengthening and/or creation of rural grass-roots organizations and improvement of farmer participation – this was the overarching strategic guideline of the project and it underlay all the actions taken; (b) the importance of the relation between development and women in the rural milieu; (c) reduction of the risks associated with drought; and (d) development of agricultural production.
The general objective of the project is to increase the income and improve the living conditions of the rural poor and residents of the semi-arid region in the Rio Gavião basin within a strategy of environmentally sustainable development. The specific objectives are: (i) support for farmer participation and development of rural organizations; (ii) financing and technical support for building small-scale on-farm irrigation works; (iii) improved social conditions and local infrastructure for the education of young people by building new schools and awarding study grants, building cisterns for home use, building roads and providing rural electrification; and (iv) higher farm and non-farm income for families by means of training, extension, marketing support and credit.
The basic ecosystem of the region is the sertão2, with its characteristic vegetation the caatinga3 . The climate is essentially tropical, although it ranges from humid to semi-arid. Small holdings are the rule in the project area (85%), and tenancy is well defined. Ninety per cent of holdings are smaller than 100 ha; a survey conducted in 1998 found the average farm size to be on the order of 30 ha. There are no community-held lands.
The small production units draw on family labour, produce crops for self-consumption (maize, beans and cassava), and raise livestock on a small scale for the market. Processing of agricultural raw materials is very common in the area, e.g. cassava flour, distilled sugar cane syrup, solidified sugar cane syrup, soft and hard cheeses, and rolled tobacco leaves. Non-farm revenue – from temporary migration of males to large urban centres – coupled with other local income and social security (pension, etc.) supplement area residents' farm revenue.
Roughly 65% of the families lived below the poverty line, with an annual family income of less than USD 1 500. Women play a crucial role in this harsh environment, performing tasks such as ensuring the water supply for the household and heading up the household when the men migrate temporarily to the cities.
According to the project organization set forth in the original design, the Regional Action and Development Agency (CAR) would be the implementing agency. The project execution unit (PEU) would coordinate the field work of over 30 professionals from the expected main partners: Bahia Association of Community Schools and Rural Families (AECOFABA), National Microenterprise and Small Business Support Agency (EBRAE), State University of South-Western Bahia (UESB), and the Bahia Development Bank (DESENBANCO).
Additionally, the appraisal evaluation called for a complex management and coordination structure made up of a directing council, an interinstitutional coordination committee, municipal coordination and implementation committees, and a consultative committee.
The actual organization of the project was modified significantly vis-à-vis the design, both internally and in terms of the partner and coexecuting organizations. All the management and coordination structures described in the preceding paragraph were not implemented.
The PEU currently comprises a team of eight professionals and technicians and two secretaries (who are staff members of the CAR), plus a team of 32 professionals and technicians and seven administrative assistants, distributed among five field offices. Project execution is conducted through contracts with the following partner organizations: AECOFABA, National Agricultural Research Board EMBRAPA), SEBRAE, National Rural Training Agency (SENAR), BAHIAPESCA S/N and Bank of the Northeast of Brazil (BNB); AECOFABA, in turn, has subcontracted the Rural Cooperative of South-Western Bahia (COOPERSUBA).
The project has strengthened the 80 existing community associations and established 122 new ones. These 202 associations enjoy legal status, regular attendance at meetings and monthly contributions from members. Over 190 groups have been set up within the associations to provide training in agricultural practices and group work; 153 special interest groups (both individual and group, e.g. commissions and microentrepreneurs) have been set up as a result of this support for rural organization. There are also 15 groups specifically created by – and made up of – women.
The project supported the construction of 3 680 cisterns for home use, medium-size reservoirs for each of 120 communities, seven artesian wells and 14 centralized water supply systems. It also improved the distribution of and access to electric power, with 2 492 km of lines, home connections and the distribution of solar energy kits that made it possible to provide services to 3 400 families. Other small-scale works that had considerable social impact were the construction of 36 community laundries, 67 residential sanitary services and 60 community watering points for livestock.
Livestock activities focused on the area's three main species: cattle, goats and sheep. Aside from the improved water supply mentioned above, the production of forage was the most significant outcome.
Other activities in this sector involved the use of credit to purchase animals.
In the self-supply/subsistence subsystem, implementation focused on enhancing the productivity and processing of cassava for human and animal consumption, including the post-harvest processing typical of this crop. Aquaculture – promoted under the project – is a new activity in the region and is possible thanks to the innovation of reservoirs in the arid North-East. The project is working with 144 families in 48 communities in the processing of raw materials, and with 29 collective microenterprises that involve over 855 area residents (39% women), currently at various stages of establishment.
A total of 1 739 credit operations have been carried out, principally for investments in forage, purchase of breeder stock, livestock management infrastructure and forage chopping machines. Credit has been part of the process of technical assistance and group management under very close supervision.
Some 1 400 training events have been conducted under the project, with an average of 23 participants per event. Over 7 300 participants (32% women) have attended 539 courses on rural organization and management of associations; 40% of them (2 968) also attended courses on rural administration. The project's work with the family farm schools (EFAs) has brought an additional 238 boys and girls into these schools; thanks to the study grants provided, post-primary education has now been extended to 483 young people between the ages of 13 and 17. The two schools in Licínio de Almeida and Mortugaba have been expanded, and a new school was opened in Anagé, thus creating an additional 238 school places in the area.
The objectives as set are attainable by the project, considering that thus far nearly 9 000 families have organized in community associations or special interest groups; some 5 000 area residents have been introduced to improved farming practices, inputs and modalities; and some 3 000 area residents have received training in new areas of agricultural entrepreneurship (2 968) and/or in improved food processing techniques (3 180).
The following negative aspects have had or are having an impact on the project's efficiency, i.e. the cost-effectiveness of attaining its objectives: (a) the delay in implementation and the duration of the first phase (over three years), with weak results and constant changes in direction; (b) the location of the coordination unit very far from the project area; (c) the delay in the launching of some very important components, such as microenterprises, which is still at an early stage and has entailed considerable costs; (d) the non-implementation of other important components, such as marketing; and (e) the weak results obtained in the irrigated crops sub-component.
The strong points include: (a) the appropriate construction and intensive use of infrastructure built; (b) the results of the cassava, forage and livestock programme; and (c) the operation of the credit programme by the BNB.
The project implementation cycle is currently at a stage of maturation, consolidation and fine-tuning of the implementation actions. As a result of the delays encountered and the activities that were not implemented, and in line with good administrative practices, funds are currently available to cover a two-year extension that could include the necessary activities and thrusts of work.
Moreover, there are a number of areas for potential improvement in the project's effectiveness.
Impact on rural poverty
The reservoirs have benefited over 7 850 families; the washing points and water points for livestock – which no longer contaminate the water supply – have yielded additional benefits for families. More than 9 350 families have benefited from investments that have boosted the production and productivity of their small herds of cattle, sheep and goats.
Among the intangible individual and community assets are the new technologies applied and adopted for cassava, forage, livestock management, vegetable crops, aquaculture, water resource management, and post-harvest processing. The knowledge and skills imparted and acquired during training – which rural residents value – is another key asset generated by the project. The attendance at the courses is an indicator of its magnitude.
Family nutritional status has improved overall thanks to the greater availability and better quality of drinking water in the home. Other nutritional improvements may have been generated by the increased volume of food produced in the countryside and higher milk production as a result of better feed and drinking water for livestock; in some areas of the project, the growing of vegetables has enriched the rural diet.
The greatest impact on food security was produced by the work done in the growing and processing of cassava, a staple of the diet and economy of area families. Agricultural revenue may improve with the expansion of the livestock herd and increase in its productivity, which is occurring as a result of the adoption of a number of technological advances. An outgrowth of the concept of food security, which bears special importance in this region, is that of water security, in other words, the ability of homes and communities to ensure the supply of drinking water for basic minimum consumption for everyone at all times. In this regard, the project has achieved its greatest impact and greatest coverage.
The health conditions of the population have improved with the increased availability of safe water provided by the cisterns and the 120 reservoirs distributed across the project area. Children, in particular, have benefited from this action.
The work with the EFAs has created 238 additional spaces in these schools, providing study grants to a total of 483 boys and girls. Among the young people, girls in particular have benefited – in the best of cases, they only receive the first four years of basic education. The number of girls enrolled in the EFAs has risen from 40 to 111.
The negative environmental impact generated by the project thus far is not significant and is easily reversible through simple measures that have begun to be implemented. It will be necessary to continue observing and analysing the development of livestock herds, especially goats, and their impact on the native vegetation " caatinga" . In general, however, the project should pursue a more pro-active policy in relation to environmental conservation.
Women's presence in all areas of work and training under the project is on average 30% or more, in accordance with the projections of the appraisal report; in several cases, women have become the main players in the activities and groups promoted by the project.
The current situation of the project's sustainability does not yet provide a guarantee for its future course. Community development is at an intermediate stage of growth and must still be consolidated and allowed to mature; this will gradually reduce the paternalist vision of the project, allowing organizations to gain autonomy and build self-esteem. The project's linkages with local society, or rather its relative isolation, is the main drawback for the future sustainability of project actions.
. Viewing all these elements together, it can be said that the project has had a promising and favourable impact on reducing rural poverty in the Gavião River region.
Performance of the partners
The appraisal report did not adequately identify and assess the partners in terms of organization of project execution. The proposed PEU was too small and was expected to work with several decision-making and coordination levels that were relatively complex. The organization identified to handle all the field work and provide social and technical assistance in a decentralized fashion (AECOFABA) had no previous experience in this area. Other partner agencies identified in the design were excluded at the outset. This led to excessive complications and delays in implementation.
The cooperating institution was UNOPS, which has kept the same supervision officer since project start-up, conducting many missions and drafting the respective reports during the implementation period.
The reports show a very good knowledge of the project, the technical team, the institutional and political context, and the technical aspects involved in the productive components. The general strategy of facilitating implementation did not question, however, the adaptative changes or other, very significant changes that have been implemented.
Performance of the Government and its institutions. It was not necessary to set up any new institutional structure to implement PROGAVIÃO. The project's central coordination was placed under the CAR, a specialized agency of the State of Bahia experienced in carrying out projects with governmental or multilateral financing.
Three important government agencies took full part as coexecuting agencies since the outset of the project: EMBRAPA, SEBRAE and SENAR. In general, the contributions of the State of Bahia and other counterpart agencies (e.g. BNB) have observed the time frames and amounts required in the loan agreement. Administrative procedures have kept pace with project needs.
Performance of NGOs and grass-roots organizations. The project has wagered heavily on an NGO specialized in the development of rural alternative schools (AECOFABA), which existed before implementation and has the status of coexecuting agency for PROGAVIÃO. Its linkage with the project was focused in three areas, of which the first two had strategic value for attaining the proposed objectives: (a) the operation of three family farm schools (EFAs) in the project area, and the possibility of providing basic education to poor rural children; (b) the provision of middle-level technicians trained at its schools for the technical teams for field work; and (c) administrative facilitation for hiring all field personnel.
BNB is the coexecuting agency for the credit services component. It has developed very efficient working arrangements that are decentralized geographically and facilitate access of rural producers to credit. Efficient cooperation procedures have been put in place with the project to manage credit.
General evaluation of the project and conclusions
Design and implementation. The major thrusts of the strategy as established in the design and in the appraisal report were adequate and effective in terms of IFAD's mission and institutional vision, and ensured appropriate participation by the rural poor and other partners. However, the organizational structure established was not viable and required numerous changes in implementation, in the planned coexecuting agencies, in the management structure and in the spaces for farmer participation in the general orientation of the project.
Impact on social capital and empowerment. After an initial implementation phase of roughly three years, with a clear focus on implementation of the scheduled works in order to produce visible results, the project emphasis was shifted to community organization and strengthening of rural participation, the gender perspective, and inclusion of families in various activities.
This second stage, which has lasted a bit over three years, has shown progress in terms of better organization and participation.
The mission found that the project is now in a phase of consolidation and fine-tuning of implementation in order to attain its original objectives. Additional time is required in order to consolidate the recent, positive trends, include more beneficiaries in the project organizations and benefits, and better situate communities in the local and regional setting.
Impact on water security. The project measures to achieve water security for producers and communities have all been a genuine success. More water is now available for use in the home and for livestock, thus facilitating work in the home and alleviating the work load, especially of women and young people.
Impact on food security. The greatest impact on the system of self-consumption of the family farm unit was produced by the work done in the growing and processing of cassava, a staple of the family diet. Agricultural revenue may improve with the expansion of the livestock herd and increase in its productivity, which is occurring as a result of the adoption of a number of improved technologies.
Gender. In order to identify women's specific needs and meet their demand for goods and services, participatory planning arrangements need to be fine-tuned so as to allow for differentiated identification of the needs of each interest group in the community, and the insertion and prioritization of these needs within the work plan.
Technical staff. The devotion, expertise and enthusiasm of the field staff are undeniable. Even so, responsibilities need to be redistributed among field technicians and social assistants; technical staff should receive training in participatory planning and evaluation methodologies; and women should gradually be included more in project management.
Monitoring and evaluation. Work remains to be done in this area, especially in harmonizing criteria for system inputs; verifying, harmonizing and expanding current data; and following up on the baseline studies (profiles). The SACC system of the CAR may be a very valuable programme to facilitate this work.
Training and education . No exact information is available on the effectiveness of the training provided under the project. Under the EFA support component, it is necessary to verify if the EFAs are now better able to offer more and better services to the communities and young people they serve, and to verify the impact and changes obtained in the management of production by the 230 young people and their families who have graduated under this arrangement.
Microenterprise development. The microenterprise component and the Microenterprises and Artisans Support Fund (FAMA) are in the initial phase of implementation. A major effort – that could be characterized as preinvestment for the component – has been made in the training and organization of entrepreneurial groups in various phases of technical assistance, such as market research, definition of artisanal-industrial processes, analysis of legal provisions that are the portals of entry to the most significant markets, health regulations, etc.
The sustainability of the project achievements is not yet ensured, and an extension of the implementation period is required so that the communities can continue their processes of learning and expansion of social capital in order to strengthen their autonomy and capacity to remain active in the post-project phase.
Performance of the partners. It is necessary to conduct a joint assessment with each of the partner organizations and to adjust or modify the respective strategies in light of the results achieved and the project's objectives in order to ensure the effectiveness of the last phase and shore up or extend the achievements sought.
Insights and recommendations
It is recommended that the project be extended for an additional period of two years. Resources are available from the original agreement amount – both from the IFAD loan and from the counterpart contribution of the State of Bahia – to finance such an extension.
The country has recently witnessed a period of sweeping change at the level of the federal government, which has assigned high priority to support and integration of the poorer social sectors, family agriculture, and those regions of the country (including the North-East) with higher levels of poverty and difficulties in attaining sustainable development. Accordingly, the project needs urgently to be redefined in an effort to seek greater local, regional and social integration and to overcome the marked isolation that still characterizes it.
The development of rural community associations and effective participation by their members should be strengthened, promoting activities for: (a) territorial integration of communities – at both the local and regional level – to allow them to participate in the municipal committees and in the various projects under way in the area; (b) extension of the project actions to most of the families in the communities currently covered by the project; and (c) various actions in the areas of productive development, post-harvest processing and marketing, especially suited to collective action.
To deepen its community work, the project should revisit its structure of technical human resources and reallocate them to the new tasks while retraining staff in accordance with the new guideline that is to be prioritized.
Some specific areas of the project should be reviewed and reformulated, specifically gender, environment and M&E activities. In other activities, e.g. marketing and post-harvest processing, all the steps called for in the formulation are pending. In line with this action, specialized technical staff should be proposed for each of these areas to assume responsibility that today does not exist or is assigned to coordinators having various other responsibilities.
During this final stage, the technical staff of the project and of the coexecuting agencies should receive additional moral support and training to offset the fatigue and lag in expertise, as well as the significant deterioration of their real incomes.
Joint assessments should be conducted with the institutions associated with execution in order to adjust or formulate the respective strategies in light of the results achieved and the project's objectives.
Looking back on project implementation, IFAD did not give proper consideration to the modifications introduced in execution vis-à-vis the proposal as originally approved. Although there was close monitoring by the cooperating institution (UNOPS), the approved modifications took on a continuous and sequential character, losing sight of the deep qualitative changes resulting from their accumulation and the strategic digressions that ensued.
Future outlook. A second stage limited to the same beneficiaries would probably not yield the same levels of effectiveness in terms of marginal outcomes and, moreover, would not be equitable vis-à-vis the extensive requirements of other municípios in the region.
PROGAVIÃO has confirmed the forward-looking strategy of combining public and private efforts to address critical infrastructure issues that have a key impact on the quality of life of the rural poor.4
At the same time, it has pursued actions to empower the rural poor by organizing communities and ensuring their participation in oversight and ownership of these processes of sustainable development.
Any new project should intervene in an appropriate area of Bahia having a baseline situation very similar to the one encountered at the beginning of PROGAVIÃO. Under such conditions, it would be possible to tap the project's positive implementation experience, as well as the institutional capacity and technical human resources generated during the period.
1/ The interim evaluation mission was composed of: Mr Carlos Pérez Arrarte, Mission Leader, expert in production, extension and marketing; Mr Edson Teófilo, expert in policies and institutions, and community organization; Mrs Myra Speelmans, expert in microenterprises, organization and management, M&E and gender; and Mr Hernando Durana, expert in rural infrastructures and environment. The Mission remained in the State of Bahia from 11-30 May 2003. Mr.Paolo Silveri, Evaluation Officer of IFAD's Office of Evaluation joined the mission at the beginning and at the end of the field work, and the Supervisor of UNOPS for PROGAVIÃO, Mr Rodolfo Lauritto, joined the Mission in its first week of work.
Hills Leasehold Forestry and Forage Development Project
The direct transfer of assets to the poor is an innovative approach to poverty alleviation. Aiming to solve the problem of common access to forest areas, 40-year leases give 1,800 household groups user rights over plots of degraded forest land totalling 7,400 hectares.
Once restored, the forest areas are a rich source of fodder, timber, and fuel as well as trees and plants that the groups use and sell. Goat ownership has increased from an average of two to five per household, as has revenue from goat sales (to US$ 100 per household per year). Income from grasses, grass seeds and other forest products is now significant (up to US$ 70 per household per year). Yet, weak market linkages and inadequate information concerning demand and market prices have limited sales in some areas.
The 120 leasehold inter-groups and 18 multi-purpose cooperatives created during the project have been instrumental in tackling these issues due to their strong bargaining power and success in creating market outlets. These grassroots organisations need further strengthening to ensure long term sustainability, however.
Future initiatives also need to pay greater attention to local traditional knowledge concerning land management, plants and animal husbandry and to involve user groups in research and decision-making processes.
Smallholder Development Project in North Lower Guinea
Despite its agricultural potential at the beginning of the 1990s, the forest region of Guinea had a food production deficit. Nationalisation policies between 1958 and 1984 had resulted in individualist, predatory subsistence farming.
Later, with liberalisation, IFAD sought to provide sustainable technical and organisational solutions to rural development problems. In seven years, the project achieved an increase in agricultural production although expected results in terms of environmental conservation and reaching the most disadvantaged were disappointing. The second phase of the project, beyond ensuring the participation of the poorest, needs to guarantee that long-term benefits will actually reach all intended target groups.
Netrakona Integrated Agricultural Production and Water Management Project
Project Completion Evaluation
There is evidence that sustainable changes have been brought about by the project. Assessing the effects of new technology and training, an impact survey used by the evaluation shows that 85 percent of respondents now use the skills and knowledge they received from training and will continue to do so.
Farmers have developed orchards and vegetable gardens and are growing new types of vegetables, which provide increased yields and higher incomes. Between 1995 and 2000 the amount of land used to grow vegetables increased by 186 percent from 2,750 to 8,950 hectares.
Ten years ago cauliflower and radishes were not available in local markets; today, a surplus is sold as far afield as Dhaka. In addition, new training and community centres facilitate communication on marketing issues or availability of social services.
However, implementation was far from participatory: greater expertise in participatory development and group mobilisation is needed in future projects of this kind. Equally, the potential for livestock development should be given higher priority and where NGOs are involved in government projects more effective coordination between the two is vital.