Agricultural Development Project (2007)

Georgia  
April 2007

Completion evaluation 1

Introduction

The economy. The ADP became effective in 1997 with the country still suffering from the abrupt withdrawal of Soviet Union capital at independence in 1991 and the collapse of the centralised economy. At the time of project design in 1995/6, industrial output had fallen to around one-sixth of its 1989 level and inflation was rampant. The United Nations Development Project (UNDP) National Human Development Report of 2001/2 estimated that 45 per cent of Georgian households were living below the poverty line, at which time the average per capita income stood at around US$680. It was not until the removal of the Shevardnadze government in 2003, that measures were put in place to address corruption and liberalise the economy.

Agriculture. Hit by the lack of investment and markets and the emigration of labour, the agricultural sector effectively reverted to subsistence farming during the 1990s. The share of agriculture in Gross Domestic Product (GDP) fell from around 50 per cent in 1990 to around 16 per cent in 2004. The unofficial land privatisation following the break-up of collective farms was regularized, with priority given to existing farmers and other rural residents. The IFAD Country Strategic and Opportunities Paper (COSOP) of 2004 saw the major challenge in rural areas as ‘the transformation of the new recipients of farmland from farm workers to decision-making farmers'.  

Rural finance. The rural finance landscape of Georgia is represented by commercial banks, credit unions and MFIs, with the National Bank of Georgia (NBG) acting as supervision and regulating agency. Commercial banks have started to move into rural microfinance, but credit unions are effectively the only institution with the potential to reach vulnerable groups.

The project. The ADP constituted IFAD's first involvement in Georgia2. It was designed by the World Bank/International Development Agency (WB/IDA), with IFAD agreeing during the negotiation phase to co-finance the components for credit unions and land registration. Total project costs were calculated at US$26.3 million financed largely by an IDA loan of US$15m and an IFAD loan of US$6m. The IFAD loan became effective in August 1997. Project completion was scheduled for April 2002 and progressively extended until 30 June 2005. The project financed five components: Credit to Enterprises; Credit Union Development; Land Registration; Development of Agricultural Services; Project Management.

Objectives. The overall objective of the IFAD-funded components was the reduction of the poverty level and food vulnerability of rural households. Specific objectives included the establishment of a network of village-based rural financial intermediaries and the emergence of an active land market. The project area of the credit union component covered the entire country, while the Land Registration component concentrated on two districts only. The target group comprised rural households with low incomes, typically with less than a hectare of land and an annual income of US$200-400.

Components. The Credit Union component aimed to support the development of up to 120 sustainable credit unions (CUs), with funding provided for loans to members and for initial costs, and to provide the basis for the legislative regulation of CUs. The aim of the Land Registration component was the establishment of land registration offices in two districts and the registration of 130 000 land ownership titles, with a view to countrywide replication of the scheme. Under the Credit to Enterprises component, a line of credit was provided to commercial banks to finance selected private enterprises, the majority of which were expected to be among the agro-processing companies that make up three-quarters of all enterprises in the country. Under the small Agricultural Services component studies were to be made to assist in restructuring agricultural services and identify appropriate investment programmes. The responsibility for implementation rested with the Ministry of Agriculture, Food and Industry.3 A Project Coordination Unit (PCU) was established to manage planning, budgeting and procurement as well as project monitoring and evaluation (M&E).

The evaluation. The evaluation aimed to assess the relevance of project objectives to the rural poor and to IFAD strategies, the extent to which these objectives were achieved, the sustainable impact of the project and the performance of the partners involved.  An evaluation mission was conducted from May 15 to 8 June 2006.  The mission held discussions with the relevant government ministries, the National Bank of Georgia, WB/IDA and other donors, microfinance institutions, project officials, staff and clients of the Land Registry Offices, and the management and beneficiaries of ten credit unions. Field visits and mini-surveys of beneficiaries were carried out for both IFAD-funded components, which constitute the chief emphasis of the evaluation.

Project performance

Project rationale. The ADP placed appropriate emphasis on re-building the rural economy after the cessation of Soviet capital investment. The establishment of a viable rural finance system and the setting-up of an efficient land registration system addressed basic needs in rural areas where such systems did not previously exist. The priorities of ADP were confirmed in the IFAD's subsequent sub-regional and country strategy papers.4

CU component design. Targeting mechanisms were set out to ensure that the CU component would reach the very poor, including the stipulation of 30 per cent CU membership for 'vulnerable' households, although the only logframe indicator relating to poverty impact was 'access to credit'. No survey of regional credit unions was carried out and the CU model was to be developed through 'learning by experience', placing a heavy responsibility on the Credit Union Development Centre, set up under the PCU to manage the component. Sustainability was defined at appraisal, but no relevant statistical targets were put in place and the exit strategy was not adequately developed. The start-up grant of US$3 000 was large enough to cause the formation of CUs solely for the sake of obtaining the grant; in some cases, the credit unions existed on paper only.

Changes during implementation. The Mid-Term Review (MTR) recommended a reduction of targeted numbers of CUs, from 120 to 55, with an attendant re-allocation of funds. The share of the CU component in the total budget thus fell from 31 per cent to 25 per cent while that of the Land Registration component rose from 20 to 25 per cent. The MTR revision reflected the unsatisfactory implementation of the CU component and allowed for the concentration of resources on sustainable CUs. The project completion date was extended three times for further institutional development in the Land Registration component and the restructuring of project-supported credit unions.

Disbursement. Around 93.0 per cent of the project funding of US$21.5 million was disbursed, including the same proportion of the IFAD loan. The principal and interest repaid by the credit unions was deposited in a revolving fund which, together with the Insurance Fund, contained nearly US$2 m in June 2006. The sum has been earmarked for credit union support under the RDP.

Credit Union component. A total of 164 credit unions were created in the first two years of the project. This over-rapid expansion was caused by politicisation of the process and resulted in widespread fraudulence and mismanagement. The financial situation of CUs was revealed in 2001 when the majority of CUDC loans were scheduled for repayment. By 2002, loan repayment performance had reached a low of 30 per cent. Auditing was not carried out, the monitoring of loans made by CUDC was weak and performance standards were neglected. Managers of credit unions were poorly trained and many credit unions were essentially structures created by village elites to harvest donor money. There was little emphasis on savings mobilisation or sustainability.

These developments took place against a challenging background which included the Russian financial collapse of 1998, severe droughts and floods in 2000 and 2001, frequent staffing changes in the relevant ministries and allegations of corruption in high places. The result was a crisis of confidence within the Government and the country at large, the withdrawal of the Finance Ministry's representative from the credit committee and a consequent two-year hiatus in implementation. Of the 164 CUs, 35 were liquidated or merged and 71 were taken to court. The underlying causes of the collapse of the network were addressed by a fresh management team in early 2003 through the restructuring of loans and the re-commencement of lending. At project completion, there were 58 functioning CUs, three more than the MTR target, but one year later the number had fallen to 39. 

Although the demand for CU membership has remained strong, actual membership fell from 12 000 at mid-term to around a quarter of that currently, with three CUs accounting for nearly 20 per cent of the total. The pro-poor quotas were abandoned after resulting in high delinquency rates. Mobilisation of savings increased up to 2001 but fell sharply thereafter, leaving the CUs dependent on project credits. At completion, only around 40-45 per cent of the total loan portfolio of the 15 top-performing ADP-supported credit unions was financed by savings, less than half of the internationally recommended benchmark. In 1998-2000, 80 per cent of loans were agricultural, but this led to repayment problems and the proportion has fallen to around 60 per cent, with successful CUs turning to commercial loans. Interest rates paid by borrowers ranges from 20 per cent to 42 per cent annually.

Land registration component. The component established modernised land registration offices in two districts close to the capital which have registered around 170 000 land parcels and issued 155 000 land titles, 85 per cent and 119 per cent of appraisal targets. Aerial photography was carried out on 2 000 sq. km of the selected districts and 10 000 sq. km elsewhere. The ground survey of land parcels was conducted by private survey companies supported by the project in the form of technical assistance. Savings were made by means of efficient surveying methods, with the average cost per parcel reduced by almost half. To these savings were added the budget increase at mid-term, enabling the component to refurbish and computerise 11 regional and 37 district registries countrywide. Over 200 staff were trained in legislative, managerial and technical aspects of land registration resulting in the emergence of an extensive cadre of officials with geodetic and cartographic skills. Project staff contributed to the first manual of registration, based squarely on the experience of the two supported district offices.

The component was implemented originally through the State Department Land Management, set up in 1997 with technical assistance provided by the project. It since been replaced by the National Agency for Public Registry (NAPR) in the Ministry of Justice, a transition physically and financially supported by ADP.  NAPR currently acts as an independent and self-financing land registration and cadastre agency coordinating 67 offices countrywide. Around 4 million land parcels have been registered in different parts of the country.

Credit to enterprises component.  A total of 48 loans were made for investment and working capital in agribusiness enterprises, to a total value of US$8.56 million. Two-thirds of supported enterprises were made up of canneries, wineries, livestock farms and hazelnut processing plants. Demand for medium and long-term credit was high and the repayment rate was satisfactory. Repayments were placed in a revolving fund from which additional loans were made until its transfer to the state budget. Despite the bankruptcy of two participating banks, the government eventually received a sum greater than the value of the initial credit line, with interest payments more than compensating for the deficit on principal. Over 90 per cent of sub-projects were calculated to have financial rates of return of around 25 per cent at mid-term, well in excess of the projected rate of 15 per cent.

Agricultural services component. Studies were conducted in the areas of irrigation development and the agricultural research and extension system. These were used in the design of two WB-financed projects, the Irrigation and Drainage Community Development Project (IDCDP) and the Agricultural Research, Extension and Training Project (ARET).

No viable CU network. An effective CU network was not achieved under the project, with the actual figure of viable CUs now at around 15-20, 13-17 per cent of the original target or 27-36 per cent of the revised target. What has been achieved, however, is official recognition that sustainable village-owned CUs are feasible and desirable, given the appropriate management approach, training programmes and level of commitment. The existence of two or three successful CUs and a further 10-12 with reasonable prospects of sustainability, the emergence of an embryonic CU association and the corpus of lessons learned mean that the component can by no means be written off as a failure. The economic and political contexts posed formidable challenges to implementation in the early years of the project.

Increased liquidity in land markets. Previous land registration procedures were lengthy and expensive, taking up to two months and involving extra-legal charges. This has now been reduced to six steps, which can usually be concluded in a single day on payment of the official fee only. There has been a marked increase in land transactions in the two pilot districts and land values have increased sharply, although limited collaboration among the eight different donors involved in land registration schemes hindered the development of a uniform system.

Agricultural credit system. The Credit to Enterprises component fulfilled its main objective by increasing credit flows to rural areas through the development of a rural commercial credit system using commercial banks. The loans made under the component constituted a significant increase in medium-term lending to the agro-processing sector, which essentially did not exist at the start of the project. The individual loans have been successful in providing rural employment, increasing agricultural output for export and achieving a satisfactory rate of return.

Poverty focus. CUs agreed that the bulk of their membership was drawn from 'average' and ‘poor' socio-economic categories, which accounted for 65-90 percent of village populations and 80-90 per cent of CU membership. Loans to better-off families were rare and to the poorest almost non-existent after the first two years. The consensus among the donor community, the microfinance sector and former project staff is that membership of credit unions will be dominated by the 'economically active' population, a reality apparently unforeseen at the time of project design.

Limited integration of components. In essence, the ADP was made up of four distinct components having only a management structure in common. The lack of zonal planning meant there was no integration between the Credit for Enterprises component and the CU component. Nor did the two components co-funded by IFAD have any close connection, with no credit unions established in the two districts selected for Land Registration activities. The small Agricultural Services component has the appearance of being tacked-on to a project with other priorities.

Assessment: relevance. ADP addressed significant constraints to rural development, in particular the lack of access to rural credit and of functioning agro-processing enterprises, but the design of the CU component was flawed by insufficient attention to capacity building, sustainability factors and exit strategy. There was also no clear rationale in terms of IFAD's comparative advantages for the Fund's involvement in the Land Registration scheme. Security of tenure for smallholders and the use of their land as collateral were included among the component objectives but were secondary to the facilitation of land transactions..

Assessment: effectiveness. The Credit for Enterprise component achieved its objectives in terms of the emergence of profitable enterprises and influence on the banking sector. The Land Registration scheme greatly accelerated the registration process and substantially assisted in the emergence of an active market in land. However, the Credit Union component failed in its stated objective, even while setting important precedents and, eventually, influencing government attitudes.

Assessment: efficiency. Direct beneficiaries of the ADP include nearly 50 enterprises, over 12 000 CU members who gained access to retail financial services for the first time, and over 100 000 landowners who received land titles or mortgage registrations. The Land Registration component was efficiently managed, allowing for expansion of the project to new areas, but donor harmonisation was weak. Efficiency ratios calculated for CUs reveal acceptable expenses/portfolio ratios, improving (but still high) expenses/assets ratios and very high staff/client ratios. The CUDC management team was overstaffed and underskilled and no component resulting in an implementation hiatus of nearly two years and 70 court cases can be considered efficient.

Performance of IFAD. The goal of building a CU network from scratch was courageous and may yet prove a sound development option. A more realistic approach to targeting, a clearer insistence on the elements of sustainability and a consistent IFAD presence on supervision missions would have been desirable. No major problems with the disbursement of funds, procurement or contractual arrangements were noted.

Performance of Government. A supportive legislative framework was established by the enactment of a Land Registration Law in 1996 and a Law on Credit Unions in 2002. Official support for the project has been fairly consistent, although the crisis in the CU component provoked the MOF to cause the suspension of the loan facility. CUDC was obliged to step in as an on-lending institution when the arrangement with MOF and the commercial banks failed to materialise, which it did without proper training or appropriate legal basis.

Performance of National Bank of Georgia. The partnership between ADP and NBG worked well in the preparation of the enabling legislation, and NBG's assistance to credit unions in meeting prudential regulations is appreciated by credit union managers. NBG regulations generally correspond to international best practices. The 2002 Law is acknowledged to have certain defects, among them the prohibitive expense of external audit.

Performance of the World Bank. The collaboration between WB and IFAD continues under RDP. IFAD is thereby involved in larger programmes with greater resources, while WB's influence in terms of policy dialogue is reflected in the close relation between ADP activities and the developing legal and institutional framework. Supervision of the project was regularly carried out by the WB, latterly with an IFAD representative on the team. WB supervision missions were slow to identify the causes of the crisis in CUs and to take preventive measures.5

Impact 6

Physical assets. The main impact under this heading has been the emergence of an active land market, the first beneficiaries of which were small landholders able to get a fair return on the sale of unutilised land. Increases in the price of newly-registered land were intensified close to the district capital, with some plots now selling at up to five times as much as three years ago.

Agricultural productivity.  Around 60 per cent of loans, with a total amount of US$1.36 m, were made by CUs for agricultural purposes. This represents a direct investment of US$7 000-10 000 in 150 villages. From 50-100 ha of new or revived cultivation per village may have been financed in this way, but no figures for increased agricultural production are available at village level and no impact indicator was included in the logframe. Some agricultural diversification was noted, for example the commercial cultivation of strawberries and greenhouse vegetables in new areas.

Food security. Some CUs have made emergency loans to very poor households, but the impression is that families affected by food insufficiency rely on relatives in the lean months. Among CU villages visited by the mission, estimates of food-insecure households varied between 5 per cent and 25 per cent, for the most part substantially below the national average of 29 per cent. Some CUs visited described a proportion of the communities they served (up to 10 per cent) as 'hungry', but these households did not constitute the target group of CUs after the first two years of the project.

Human assets. At least 2,000 CU loans were used for consumption purposes, with medical and educational expenses among the first priorities. At one CU, 40 per cent of all loans were used for health and education purposes. Of the households interviewed in 2003, nearly a third utilised their loans for medical treatment, educational expenses, house renovation, household goods and wedding expenses. Training was provided to CU staff in financial management, loan appraisal and book-keeping, and the emergence of well-trained staff in the land registry offices has been a notable achievement.

Social capital. Villagers stated that the CUs supported them during difficult times and gave them a sense of hope. This was borne out by the 2003 survey, in which views of overall CU operations were 99 per cent positive. CUs are often found at the heart of village life and are gradually becoming stronger self-help institutions. Women account for 50-55 per cent of CU membership and the same percentage of loans.  In many CUs, women dominate the committees, and the managers are often women.

Institutions and services. The 2002 Law on Credit Unions was drafted with the support of CUDC. New reporting forms and prudential standards were introduced and training was provided for credit unions in applying them. Under the land registration component, land registration offices using modern techniques and streamlined procedures revolutionised an inefficient system of doubtful legality. The establishment of the NAPR was a direct result of ADP initiatives.

Financial assets. More than 12 000 rural people gained access to retail financial services for the first time since independence. Over US$1.5 m were mobilized in savings and around UDS 7 m in rural lending was generated by CUs. All surveys indicate that the CUs constituted the only available source for the great majority of these loans, which enabled borrowers to initiate new income generation activities or to re-start productive enterprises relinquished through lack of finance. The household survey of  2003 showed that 28 out of 40 household used their loans primarily for business purposes. The range of income increase was wide (17 per cent to 200 per cent) but never insignificant. 

Sustainability and ownership. Project achievements in the spheres of land registration and credit to enterprises seem assured, on the one hand through the institutionalisation under NAPR, and on the other through the boost to the banking system. Of the 15 CUs for which sufficient data exists, two have achieved a financial sufficiency ratio of 85 per cent and a further six 65 per cent, but the sustainability of the remainder is problematic. The picture is more favourable in terms of 'ownership': there is everywhere strong evidence of commitment among members and managers, including community initiatives funded by members and managers working on reduced or unpaid salaries.

Innovation, replicability and scaling-up.  The major innovation of ADP was in its attempt to set up a village-based network of financial institutions in a country where there was virtually no access to formal credit in rural areas. This was a bold initiative in a country with a generally negative attitude towards cooperatives in the wake of the Soviet experience. With the benefit of the hindsight, the limitations of the project were to (i) underestimate the risks of developing financial institutional innovations under a volatile policy environment and (ii) provide limited technical assistance to support the development of CUs (the CU component was based on WB experiences in Albania, where the 'learning-by-experience' method had given positive results). Under the Land Registration scheme, the ADP registration system has been chosen as the model of countrywide replication.  This case provides an example of replicating an innovation by forging an alliance with a partner (EU) with practical experience and relevant technical expertise (through its former pilot project).

Conclusions and recommendations

Conclusions

Overall assessment.  ADP was a project implemented in difficult conditions.  The establishment of a replicable system of registration and titling was a major achievement. The lack of integration between components, however, meant that the project was no more than the sum of its parts. If the credit unions had served to finance the production of the raw material for supported enterprises, the overall impact of the project might have been more impressive.

Disturbing financial trends in CU development were overlooked by supervision missions and the MOF decided to withdraw its support rather than seek remedial measures. The result was a setback to the reputation of the CU movement in Georgia from which it is only now recovering. Yet the obstacles to sustainability are not insuperable.

The road to CU sustainability.  In order to achieve long-term sustainability, credit unions need to concentrate on savings mobilisation, loan diversification, closer attention to loan assessment and control of delinquency, and the minimum of loan refinancing and rescheduling. The best performing credit unions seek their clientele over areas wider than a single village, have de-emphasised agricultural loans, accept only members with strong repayment capacity, and are considering moving their headquarters to trading centres.

Vulnerable groups. The risk is that CUs will focus on the more attractive urban loan market and lose touch with the poor in rural areas. Particularly affected would be groups lacking in entrepreneurial initiative and unable to meet collateral requirements. The Government considers that farming households which the lack the labour or the ability to exploit their land in a businesslike way can only be reached by welfare programmes, but such programmes are presently scarce.

Alternative approaches. It is important for IFAD to explore other approaches.  An immediate objective must be to ensure that all cultivable land is productively utilised. Movements in this direction have been taken under RDPHMA, notably the setting up of cooperative ventures to provide inputs and machinery, but the success of this approach is not yet clear, also due to implementation management problems the project is facing. 
 
Markets. Alternatives to the banned Russian market are being pursued by the Government, but solutions will take time. Private initiatives are being undertaken to set up processing plants in rural areas but these are still isolated examples. Meanwhile, the issue of access to urban markets within Georgia needs to be urgently addressed. In remote and mountainous regions, roads and transport are one aspect of this problem, but there are also indications that large urban markets, particularly Tbilisi itself, are controlled through unofficial monopolies.

Partnership with the World Bank. This report stresses the advantages of partnership with WB in Georgia, while drawing attention to its problematic aspects, which include differences over the importance of targeting and a neglect of poverty impact in supervision. IFAD has also moved out from its area of expertise by co-financing the Land Registration component, probably as a result of the Fund's involvement after the main features of the project had been determined.

Recommendations

IFAD should clarify its priorities and conditions for involvement in co-financed projects. The Fund should ensure the inclusion of impact indicators connected with its concern for marginalized groups and should maintain a regular presence on supervision missions.

Measures will be required to support households that are unable to "automatically" take dvantage of interventions aiming at agricultural commercialisation. The Government needs also to intensify the development of an effective extension system and take measure to liberalise access to urban markets.

Credit unions are part of the solutions to rural poverty but not a panacea.  Measures are necessary to strengthen the economic tissue of rural areas and increase the number of potential borrowers.  These measures may include: (i) more direct solutions to the problem of uncultivated arable land, based on access to mechanised equipment for example through customised financial services; (ii) improved marketing methods for agricultural products, including attention to post-production phases; (iii) the liberalisation of urban markets within Georgia.

Future microfinance projects in Georgia and the region should learn from the principal error made under ADP, which was to allow the numerical target of CUs to become the predominant indicator. Project designs should recognise that the sustainability of CUs depends on high repayment rates and seek appropriate ways of reaching the poorest groups. CUs that have reached a certain level of assets could be offered incentives to undertake pro-poor initiatives.

Regarding land registration, the mission recommends measures to protect rural households from the effects of land speculation in areas where the demand for residential property is likely to catapult land prices beyond the means of local farmers. The priority given to village residents in the original distribution of land was aimed at guaranteeing the basic livelihoods of those depending on the land for their survival and this aim should not be jeopardised.


1/ The evaluation team was led by Mr Roger Norman, Agricultural Economist; Ms Margarita Lalayan, Rural Finance Specialist, and Mr Giorgi Badrishvili, Rural Sociologist.  The main evaluation mission took place from 15 May to 8 June 2006.  The wrap up meeting (8 June) was attended by Mr Pietro Turilli, Country Programme Manager/IFAD-PN.  Mr Roger Norman and Ms Thuy Thu Le, Evaluation Research Analyst/IFAD-OE conducted a preparatory mission in April 2006.  Mr Fabrizio Felloni, Lead Evaluator/IFAD-OE supervised the evaluation process.

2/ ADP was followed by the Rural Development Programme for Mountainous and Highland Areas (RDPMHA) and the Rural Development Project (RDP), co-funded with WB-IDA and other partners. The three IFAD loans amount to an investment of around US$25 million over a period of 15 years.

3/ Now the Ministry of Agriculture (MOA).

4/ Sub-regional Strategy Paper 1999, Country Opportunities and Strategy Paper 2004.

5/ Duly acknowledged by the WB in their December 2005 completion report.

6/ Two impact indicators only were selected for the IFAD-funded components. There was no monitoring of food security or income levels among beneficiaries, or of changes in the utilisation of cultivable land, and no baseline survey for these indicators. Partially offsetting these omissions were a 1998 Rural Poverty study and a 2003 OE Thematic Evaluation, which both contained useful household level data. The present mission conducted 34 interviews with CU members and non-members in ten villages and with a further 32 beneficiaries of the Land Registration scheme.

LANGUAGES: English

Niassa Agricultural Development Project (NADP) (2007)

Mozambique  
April 2007

Completion evaluation

Project history and design

The design process started in 1991 before the peace agreement in 1992. Appraisal was done in 1993, and in April 1994, IFAD's Executive Board approved the Niassa Agricultural Development Project (NADP) with a budget of US$ 20.1 million financed by an IFAD loan of SDR 8.8 million (~US$ 12.4 million), co-financing of US$ 4.1 million from the OPEC Fund, and a contribution of the Government of Mozambique (GOM) corresponding to US$ 3.6 million. The budget was allocated for three components plus project management, including institutional development support: (i) Agricultural Support Services (US$ 7.2 million); (ii) Roads (US$ 8.2 million of which the Organisation of the Petroleum Exporting Countries (OPEC) contributed US$ 4.1 million for rehabilitation of a primary road); (iii) Community Initiatives (US$ 2.1 million). An amount of US$ 2.6 million was allocated for a Project Facilitation Unit (PFU), monitoring and evaluation and institutional development support.  

In 1992, Mozambique was the poorest country in the world, emerging from a long period of (civil) war and the effects of oppressive colonial rule. And the situation in Niassa province, remotely located in the north-west corner, was far below national averages. At the time of design, security was a critical issue in many parts of Mozambique, and it was decided that NADP would cover only two of Niassa's 15 districts, i.e. Lichinga and Sanga which had a reasonable security situation. Niassa is Mozambique's biggest province accounting for 16 per cent of the area, but it is sparsely populated accounting for only five per cent of the population. Agricultural land is abundant. The population of Sanga and Lichinga districts is today about 140 000 accounting for less than one per cent of the national population. The Appraisal overestimated the population of the two districts, and thus the target group, by about 100 per cent. At the design stage (1993), it was assumed that 45 000 households (94 per cent of all households in the two districts) were farm-dependent and poor, thus constituting the initial target group. NADP's objective was to improve the levels of income, employment and food security of 45 000 poor family farm households. However, the 1997 population census recorded a total population in the two districts of 107 243 corresponding to about 25 000 households.

The design of the Agricultural Support Services Component included innovative features such as introduction of participatory and farmer-driven research and extension. To enhance outreach and ensure participation, 300 young progressive farmers (of whom 100 would be women) were to be appointed and facilitated as Village Extension Guides (VEGs) to assist their fellow farmers with adoption of improved crop husbandry methods. However, the VEG system was not introduced because the Training and Visit system at the time was the official extension policy. The design also included support for establishment of a Technology Support Fund (TSF) which was to be a revolving fund that would support artisans and farmers with manufacturing of simple equipment, and finance testing of labour-saving technologies. However, the TSF was not established and instead treadle mills/pumps were piloted. An intervention for Support to Rural Trade was included to provide technical assistance and training to individuals and groups involved in rural trade, and rehabilitate 115 village stores to be sold or leased to interested traders with the proceeds going into the District Development Fund. This sub-component was re-redesigned in 2000 and implemented quite differently. Finally, the design included support for establishment of pilot Savings and Credit Associations (SCAs). The target was to establish 95 SCAs benefiting 1 900 households. This sub-component was not implemented though some farmers' organisations did receive some advisory services from the extension system.

The road component had two main sub-components: (i) Primary road rehabilitation which included spot repairs and upgrading of 280 km primary road between Lichinga town and Cuamba in the south. At the time this was a dirt track but the only access to the province; the rehabilitation was to be financed by OPEC with US$ 4.1 million; and (ii) Construction and rehabilitation of feeder roads which would finance rehabilitation of 173 km of secondary and feeder roads as wells as maintenance of 300 km of secondary/feeder roads. In addition, the component would finance an environmental impact study and an economic analysis of an investment in rehabilitation of the railway link. Neither was done, however.

The Community Initiatives Component had two sub-components. (i) Village Infrastructure which would finance construction of 100 water points, rehabilitation of 16 health posts with medical equipment and furniture, and rehabilitation of 24 two-classroom primary schools including furniture and teaching materials. Rural promoters would be deployed to ensure community participation; and (ii) establishment of District Development Funds (DDFs) for funding micro-projects demanded by the local communities who would finance part of project costs. The DDF would be financed directly from project funds and indirectly from funds revolving from the support to rural traders.

Project coordination and management: At national level, the National Rural Development Institute (INDER) would coordinate and oversee implementation which would be delegated to the Provincial Planning Commission (PPC) of Niassa Province, later on the DPPF (Provincial Directorate for Planning and Finance). INDER would be facilitated through provision of computer equipment and travel budget. The PPC would be "strengthened by the establishment of a strong Project Facilitation Unit (PFU)" which would include a monitoring and evaluation unit.

Results and performance

It was highly relevant and in accordance with IFAD's mandate to provide support for Niassa province but it was also a major challenge as the capacity in the public and the private sector was extremely limited. At the time, NADP was the first major external support intervention in the province.  The design significantly overestimated the population and target group while underestimating the challenges. Therefore, potential benefits were overestimated, in particular the benefits of the support for agriculture. Assumptions regarding how much NADP could increase yields and production were unrealistic given that input and produce markets hardly existed and considering the challenges of introducing improved crop husbandry in a traditional system of shifting cultivation (slash and burn).  Though such a system to outsiders may be considered as a wasteful use of natural resources, it may for the farmers be the optimal choice given the land abundance in the area.

The agricultural support services component did improve the capacity of research and extension and for some years the outputs and services increased. However, as indicated above, the support was not implemented according to design. Output delivery was inefficient for the extension services and highly inefficient for the re-designed support for rural traders, which was poorly managed by the contracted government institute, Fund for Promotion of Small Industries (FFPI). The Lichinga Agricultural Research Station (EAL) was moderately efficient in delivering its outputs and services. However, the number of farmers covered by the services was significantly less than targeted and the impact in terms of increased yields and production is negligible. For the major crops, yields stagnated or declined during the support period. Thus, the component is assessed as unsuccessful in achieving its objectives (effectiveness).

The road component shows a highly mixed performance. Only half of the targeted primary road was rehabilitated with the OPEC funds, and the rehabilitated section needed further rehabilitation immediately after project works had been completed. The target for secondary and feeder roads was surpassed by 51 per cent, though there were numerous implementation problems due to the limited capacity and experience of contractors and implementing agencies. The rehabilitated and constructed secondary and feeder roads have provided very important benefits to a large part of the target group. Thus, in spite of efficiency problems, this sub-component was highly effective in achieving its objectives.

Under the Community Initiatives Component, the Village Infrastructure Sub-component has achieved 72 per cent more than the targeted water points, 58 per cent more than the targeted school rehabilitation and almost the target for health post rehabilitation. The sub-component is assessed as effective and moderately efficient. The support for water points stands out as a highly successful intervention, providing important benefits to about 43 000 people with a relatively modest investment. The District Development Fund (micro-project) Sub-component has no targets but is based on community demand and promoting community participation. It has funded various activities and investments, including water points, road maintenance, soccer equipment, electricity supply, and training of midwives. Implementation was constrained by lack of clear guidelines, and community participation in setting the priorities and implementing the micro-projects did not meet the ambitions of the design. However, some interventions did produce important benefits. The sub-component is assessed as moderately inefficient but moderately effective.

Economic Rate of Return: The Appraisal Report (AR) arrived at an estimated Economic Rate of Return (ERR) of 15.7 per cent by only including, on the benefit side, the incremental crop production, and on the cost side, the investment and recurrent costs of those components which are related to productive activities. Very large production increments were assumed. As actual production increments are nil or negligible, a repetition of this exercise at project completion results in a negative ERR. However, this disregards the important benefits obtained from the support for water points, secondary and feeder roads, and health and education services.

Impact on rural poverty

Overall impact: As it is impossible to attribute any increase in crop production and yields or any improved trade to the Agricultural Support Services Component, the component is assessed as unsuccessful in terms of impact on rural poverty. However, the component has contributed to improving the capacity of the research and extension systems. The support for primary roads is assessed as moderately unsuccessful while the support for secondary and feeder roads is found to be moderately successful. The Community Initiatives Component is assessed as moderately successful with the investment in water points standing out as being successful.  

Targeting: The design did not include a differentiated targeting strategy. Spatially, it appears that NADP investments in social and economic infrastructure have been evenly distributed over the two target districts, which however are better off than most of the other districts in Niassa. No special priority has been given to the poorest areas within the target districts, e.g. the lakeside. Targeting within a community has largely been left to the village leaders who not always prioritise the poorest households when identifying the beneficiaries of a specific support intervention, e.g. the goat scheme.  

Gender impact: The three components have had a highly differentiated gender impact. The Agricultural Support Services Component has had limited focus on gender issues and the outreach to women is found to be negligible. All of the 22 extension workers in the field are men who for cultural reasons seldom can work directly with female farmers. The Roads Component appears to have complied with government policy that women shall constitute minimum 25 per cent of those employed in road works, and many households including women have obtained employment and income from road construction. The Community Initiatives Component has provided important benefits for women, notably through the support for water points which has reduced women's labour burden and improved health standards. The support for adult literacy, midwives and other health and education services has also benefited many women.

Impact on Institutions, Policies and the Regulatory Framework: Partly as a consequence of NADP, the institutional capacity has been strengthened in the province and the two districts. The organisations, both public and private, involved in implementation have gained significant knowledge over the years, reflected in gradual (yet insufficient) improvements in implementation performance. NADP's participatory approach, working directly with districts, communities and Community based Organisations/Non-governmental Organisations (CBOs/NGOs), may together with other similar projects have influenced GOM's decentralisation policy and the design of other similar projects. However, there is no evidence to suggest that IFAD has used NADP as basis for policy dialogue with the government, with the view of actively influencing the decentralisation policy.

Innovation, replicability and scaling up: The key innovative design feature was introduction of participatory research & development as well as farmer-driven extension based on progressive farmers being appointed as VEGs. Unfortunately, this was not implemented because the official policy at the time was based on the T&V methodology. It remains unclear why this approach was rejected while GOM, by approving NADP, implicitly had given its endorsement. Other innovative design features, which were implemented though not fully according to the ambitions of the design, included the decentralisation of authority and promotion of community participation in setting priorities and implementing activities. These features are today being widely replicated. However, insufficient efforts were devoted to identifying and working with existing farmers' and community organizations.

Sustainability: Generally, activities which GOM considers as a core government responsibility are most likely to be sustained. Such include primary schools, health centres/posts, primary and secondary roads, and to a lesser extent feeder roads and water points. However, community participation may ensure the sustainability of water points and feeder roads. Fortunately, government resources, complemented by donor support, have increased considerably which improves the sustainability prospects for core public utilities and services. Activities, which are outside core government responsibility and which have dubious commercial viability, have limited likelihood of being sustained. Such include the rural trader scheme, which also is affected by mismanagement, the treadle mill scheme and the revolving goat scheme.

Project management and coordination

Frequent changes of project management and coordination structures have occurred since project start. The design envisaged that INDER would coordinate policy and planning with concerned ministries at the national level, but would delegate management responsibility to the provincial planning and finance directorate (DPPF) and the PFU in Lichinga as far as possible. Due to logistic reasons and problems of recruiting competent PFU staff to Lichinga, project management was mostly done from Maputo up to 2001 when the duty station of the Project Executive Director (in 2002 merged with the position of Project Coordinator) and the Financial Management staff was changed to Lichinga. One of several reasons why it took time to move the management of the project to Lichinga was that there were no banks in Lichinga that operated a foreign exchange account. Major issues throughout the project period include: (i) the limited capacity at provincial level to manage and coordinate the project; (ii) insufficient, irregular and unsystematic training of project staff and partners; (iii) frequent changes of project staff; (iv) the absence of an operational forum for coordinating activities and build consensus, also with other donors; (v) lack of internal guidelines and manuals for project implementation, such as a Project Implementation Manual (PIM); (vi) no clear mandate or terms of reference for the Steering Committee (decision making or advisory body?) and no beneficiary representation; and (vii) insufficient use of existing farmers' and community organizations.

NADP did not succeed in establishing an operational Monitoring and Evaluation system, despite considerable investments, an explicit requirement of the Loan Agreement, and recommendations in supervision reports and by the Mid-Term Review in 1999. A Monitoring and Evaluation Officer was in place only between April 2003 and February 2005, when his contract was terminated due to unsatisfactory performance. The consequence has been a lack of systematic data for indicators, ultimately leading to difficulties of monitoring, coordinating and planning project activities. Associated with this problem has been the absence of an Annual Work Plan and Budget following IFAD guidelines using Participatory Rural Appraisal and a proper LogFrame. These instruments were produced for the first time in 2003 with technical assistance support from IFAD.

Financial reporting was done according to the expenditure categories of the Loan Agreement and not according to components, interventions and outputs. Reportedly, the financial accounting system was modified around year 2000 so that it became possible to link expenditure in different categories to outputs and components. However, it has not been possible to obtain information on how much has been spent on the different components/sub-components/outputs over the project period. There is no updated asset registry specifying the economic value of items in the inventory but only a physical inventory with no values attached. Audit reports between 2000 and 2004 bring forward issues such as payments outside the objective of the project, procurement procedures in acquiring expensive IT equipment and services in 2003 for the monitoring system, erroneous booking of expenditure, and FFPI's non-compliance with the contract. A special audit of the support to rural traders through FFPI is currently being conducted and it is recommended that IFAD and GOM follow up on this.

Performance of partners

IFAD: In spite of the fact that IFAD has provided relevant support in line with its mandate, the overall performance of IFAD is assessed as moderately unsuccessful. The design was based on several wrong assumptions, in particular an overestimate of the target group by 100 per cent, which may partly be explained by the fluid and uncertain situation during the design period. However, as normalcy returned, IFAD should have revisited the design. It was too late to wait for the Mid-Term Review in 1999. IFAD allowed the project to become effective and go ahead in spite of the fact that an important condition for loan effectiveness was not fulfilled, namely the requirement for establishing effective arrangements for monitoring and evaluation. IFAD appears to have been distant, in particular in the early period. It was only very late in the implementation period that IFAD started to provide direct support to the PFU, e.g. advisory support on how to prepare participatory work plans and budgets. The re-designed rural traders' scheme appears to have been approved without any in-depth scrutiny. Finally, there is no evidence to suggest that IFAD has made active use of the NADP as a basis for engaging in policy dialogue.

At the end of the implementation period, IFAD established country presence and there is no doubt that NADP would have benefited from having an IFAD representative in Maputo throughout the entire project period. This would have helped to identify various constraints faced by the implementing institutions, and to ensure a more timely response to deviations from project design. An additional benefit could have been a more active participation in donor coordination and in policy dialogue processes.

Cooperating Institutions: In the period 1995 to 1998, the World Bank served as Cooperating Institution (CI), supervising project implementation. This was considered a practical arrangement because the Bank's missions also supervised and visited the World Bank financed Rural Rehabilitation Project. In January 1999, United Nations Office for Project Services (UNOPS) took over as the CI, in full agreement with government and the World Bank whose project had ended. Both CIs have provided timely and frequent supervision missions, however for different reasons their performance is assessed as moderately unsuccessful.

Though there is noting to indicate that the Bank's supervision has not satisfied the formal requirements, the Bank's efforts (and flexibility) to facilitate progress under very difficult circumstances were insufficient. The Bank was strict on enforcing the requirements and conditions, defined in the Loan Agreement as well as in its own operational policies but did not arrange for the capacity development support that national partners needed to deal with these requirements. Applying standard procurement rules assumes a certain capacity within government implementing agencies as well as a private sector with some capacity and competition.

Compared with the Bank, UNOPS tended to adopt a more pragmatic approach, shifting focus from procedures to outcomes and achievements. UNOPS was more flexible on project management, speeding up the process of loan disbursement and procurement. However, though UNOPS' supervision reports were more detailed than the Bank's, they failed to establish a transparent picture of physical outputs, and the expenditure on each output/sub-component/component. Stricter guidance should have been provided on how to report on physical progress and financial expenditure, setting out the actions and responsibilities of the PFU and implementation partners.

Government: Several government institutions have been involved in the implementation: (i) project management and coordination has been the responsibility of INDER (later DNPDR) through the PFU; (ii) provincial directorates (agriculture, public works, education and health); (iii) district administrations (micro-projects); (iv) EAL (agricultural research); and (v) FFPI (rural traders' scheme). The performance of most provincial directorates and the district administrations is assessed as moderately successful while the performance of the overall project management and coordination (INDER/DNPDR/PFU) is assessed as unsuccessful. EAL has delivered many of the services and outputs expected, however with limited impact, and its performance is assessed as moderately successful. FFPI has mismanaged the support (US$ 600 000) for rural traders and its performance is assessed as highly unsuccessful.

NGOs: A number of NGOs have been involved in the implementation. For water points, the international NGO Water Aid was contracted, which in turn contracted national NGOs. Generally, these NGO's have performed well and their overall performance is rated as successful.

Co-financiers: The OPEC Fund played a passive role as a co-financier but did efficiently what it was expected to do, e.g. quick release of funds. The overall performance is assessed as moderately successful.

Overall assessment of NADP

The overall rating of NADP is "moderately unsuccessful" which as explained above hides over big differences. The two major budget items, i.e. the support for agriculture and the primary road, are rated between "unsuccessful" and "moderately unsuccessful" which pulls down the project average. However, even when the components and sub-components are not weighted with their respective estimated share of total expenditure, the overall rating is "moderately unsuccessful". The support for secondary & feeder roads and community initiatives is rated as "moderately successful", with the support for water points achieving a rating of "successful". Overall partner performance is rated "moderately unsuccessful" with NGOs achieving a rating of "successful" while at the other end, FFPI's performance is rated "highly unsuccessful". IFAD and cooperating institutions obtain a rating of moderately unsuccessful. "Project Performance" (relevance, effectiveness, efficiency) scores closer to "moderately successful" than "moderately unsuccessful" thanks to high ratings of relevance.

In spite of a modest overall rating, the Evaluation Mission agrees with those Mozambican observers who find that NAPD has been an important project. At the time, NAPD was the first major intervention in Niassa and it had a pioneering role, paving the way for others to follow. Though important, it is difficult to quantify the benefits of this pioneering role. NADP, together with other support, has made an important contribution to the development of the capacity in Niassa's public and private sectors from a very negligible base. And, NADP has positively changed the daily life of many households through its support for water points, roads, and rehabilitation of education and health facilities.  

Insights and recommendations

When project design takes place in situations of extreme uncertainty, e.g. post-conflict and post-emergency, an inception review should be implemented to assess the validity of assumptions and the design, and introduce the needed design modifications early in the implementation period. The review would be done jointly with the government and have the mandate to recommend the design changes required. Preferably it should be done when the project management team is on board and has had some time (say six months) to assess the situation and the project design.

It is unusually difficult to forecast the project context when designing a project at the end of or just after a conflict (or emergency), where the situation is fluid, volatile and uncertain. When "normalcy" returns, the reality may change dramatically within a short period of time (one year), making some of the project assumptions and part of project design irrelevant.  The time from project approval to Mid-Term Review (MTR) is normally four to six years, depending on the elapsed time from approval to effectiveness, and this is too long to wait for adjusting design to the changed realities. NADP was designed during 1991-1993, i.e. just around the peace agreement in 2002. Serious implementation only started around 1996/97 and the MTR was implemented in 1999. The NADP design would most probably have been different in a number of aspects if it had been designed during 1995/96.

The Government of Mozambique should assess if there is a need for strengthening its internal systems for appraising and approving donor-assisted projects. It was unfortunate that some of the innovative design features, notably the VEG system, were rejected following the approval. This may be an indication that the government did not fully take ownership of the design process and/or that the government's internal appraisal/approval system did not sufficiently analyse the details of the proposal. Since then, the internal systems may have improved but it is suggested that GOM assesses whether the current systems are such that the experiences of NADP are avoided.

Projects in remote areas need to be managed and coordinated locally and the feasibility and problems of doing so need careful assessment at the design stage. During NADP's first five years, implementation was managed and coordinated from Maputo which negatively influenced performance. This was contrary to the design but considered necessary for a number of unforeseen practical reasons. It is, therefore, important to assess the obstacles already at the design stage and include measures to reduce the obstacles in the design (special employment conditions, communication facilities etc.).  

In remote areas with limited human and institutional capital, considerable investments in capacity development are required during the early implementation phase, and repeat-training is needed to bring new staff on board. These needs are significantly higher in multi-sector projects with many implementation partners. Though NADP had an adequate budget for training it was not utilised, and too little was done initially to define implementation rules and procedures and to train the many partners. In post-conflict and post-emergency situations, there is a natural urge to deliver something on the ground quickly, and there may be a tendency to jump into implementation without first establishing the foundation for doing so. Therefore, in such situations the effort to develop capacity and systems need to be intensive but brief. Subsequently, when implementation had started, NADP should have done more to organise repeat-training in order to bring new staff on board. Such investments may be considered as "delivery costs" and for a multi-sector project in a remote area working with numerous institutions with limited capacity, these costs are considerable but necessary for achievement of impact.

Since 1994, IFAD has obtained more grant resources for supporting the portfolio and the partner countries. In a future project similar to NADP, it would be relevant to use grants for providing intensive support for capacity development and for close guidance and implementation support, in particular during the initial stage.

Major interventions in areas of core government responsibility have performed better than small scattered interventions in the border area of the public sector domain. The investments in secondary and feeder roads and in water points, schools and health posts have made an important difference in the districts and have reasonable sustainability prospects. Performance of these interventions was considerably better than the performance of small scattered activities such as goat re-stocking, rural traders' network, treadle pumps etc. which tended to become appendices without the required management support. While there are provincial directorates which have mandate, responsibility, staff and budget for roads, and health and education facilities and services, there is no provincial directorate whose core responsibility (with related budget) is to promote goat re-stocking, treadle pumps, or village shops. Furthermore, government institutions and staff are generally not specialised in developing commercially viable businesses. When the management capacity is limited, there is a need for focus and a less diversified support menu, even under a participatory approach.

Furthermore, the NADP-supported interventions of a public goods nature have better prospects of sustainability than the NADP-assisted commercial activities because the commercial viability of the latter is uncertain. However, government needs to take follow-up actions to ensure the sustainability and benefits of some of the public goods interventions, including:

  • In the roads component, there are some unfinished works including construction of an aquaduct on the N'toto – Choule road, two bridges, and completion of Unango – Muembe road. These works need to be funded and completed in 2006. Reportedly, the budget for these investments is included in the provincial Annual Social and Economic Plan (PES) for 2006 but it needs to be ensured that it is spent for this purpose.
  • A decision regarding future ownership and management of the equipment financed by NADP for road construction and maintenance needs to be taken before project closure.
  • The support for mobile midwives has had a positive impact and is much appreciated. However, to sustain the services, the maternity kits provided to the midwives need to be replaced periodically.

Targeting strategies and selection criteria are required in design and implementation. All farm households were included in the target group leaving it to community and district leaders to select the beneficiaries of various support interventions. Occasionally, beneficiaries were not poor, and the needs of the most vulnerable households were neglected.

Design and implementation should clearly address gender aspects, strategies and resources. In doing so it is important to take into account cultural and traditional constraints and practices. For example, it may prove difficult for male extension workers to reach female farmers since interaction with the opposite gender often is seen as inappropriate, particularly for married women. Targets and objectives should also be defined, allowing for gender disaggregated M&E indicators, and progress reporting on gender strategies and targets. For example, abandoning the design strategy of appointing 100 female VEGs should have been an issue for discussion.

An effective approach to raising agricultural productivity, production and incomes in sparsely populated remote areas with land abundance has not been identified. This is a negative lesson learned and very frustrating since a major effort was invested in this objective. It remains for discussion why the efforts had limited impact and what could have been done differently to ensure the achievement of this important objective.

If the explanation is that farmers do not respond to recommendations on improved crop husbandry because they cannot access the required inputs and tools and/or because there is no market for their incremental production, then this would indicate that investments in research and extension are premature and need to await the development of market infrastructure and services. For the government, which is trying to prioritise a limited resource envelope, which is insufficient to support all areas of Mozambique with research and extension, this would indicate that priority should be given to commercial agricultural areas with a certain level of market development, while road infrastructure and livelihood investments (e.g. water points) initially would be more relevant for remote areas like Niassa. However, an alternative explanation could be that farmers have no demand for changing their traditional extensive shifting cultivation system as long as land is abundant. If this is the case, one may question the role of research and extension, at least the approach.

The capacity of District Administrations has improved significantly over the project period, and if NADP were to be implemented today, District Administrations could serve as implementing agencies for many of the NADP supported activities, while the provincial government would have the overall implementation responsibility. A District Administration of 2006 is completely different from a District Administration of 1994. With some support, most district administrations would today be capable of implementing many of the NADP activities. As compared to the current management from a provincial directorate in Lichinga town, this would bring management closer to the beneficiaries, with a number of consequential benefits.

Finally, in a possible future area-based multi-sector project, IFAD and GOM may consider to assign the overall implementation responsibility to the provincial government, instead of a national line agency, but with the Ministry of Finance or Ministry of Planning and Development providing general oversight and serving as the official national counterpart.

LANGUAGES: English

Arhangai Rural Poverty Alleviation Project (2007)

Mongolia  
April 2007

Completion evaluation

Introduction

Background. After Mongolia joined IFAD in 1994, discussions between the Government of Mongolia (GOM) and the Fund led to the formulation of the first IFAD-financed project in the country, the Arhangai Rural Poverty Alleviation Project (ARPAP).  An IFAD loan of SDR 3.45 million (approximately US$5.04 million) was approved by IFAD's Executive Board in April 1996 and the project was declared effective in the same year.  Between 1996 and 1998 the project was implemented in the Arhangai province; after the Mid-Term Review (MTR) in 1998, the project scope and implementation was extended to the Huvsgul province. Following an extension of six months, the project was completed in December 2003, while the repayment of the livestock loan will continue till 2012.

The completion evaluation mission visited the country between 1 and 30 September 2005. The objectives of the evaluation mission were to: (a) assess and document the results, impacts and effectiveness of the project; and (b) draw lessons learned applicable to the design of future projects.  The mission held working meetings in Ulaanbaatar with authorities from the Ministry of Food and Agriculture (MOFA), the Ministry of Finance (MOF), and various stakeholders, including representatives of other development agencies operating in Mongolia.  A total of 20 sums within the project area were visited by the mission members and discussions were held with a cross-section of stakeholders, including beneficiaries.  A household survey covering representatives of 180 households was conducted during the mission by the Centre for Policy Research.

Changing conditions during the lifetime of ARPAP. At the time of project design, Mongolia was a country in transition following the withdrawal of Soviet subsidies. The urban economy was in decline, and for the first half of the 1990s, this was reflected in a flow of population back to the countryside. However, the economy of Mongolia changed radically during project implementation, following the liberalisation of the mining sector, the growth in export markets for livestock products and the back-migration of rural populations to the towns.

Project features. Given the limited experience in project management of GOM officials in the mid-1990s, the project design incorporated the following features:

  • moderate number of beneficiaries with an initial phase in one aimag (province) to assess local absorptive potential and strengthen institutional capacity;
  • simplified components and interventions to address the needs of the rural poor in a gradual manner;
  • beneficiary involvement in defining the composition and number of the herds, and strong awareness of the terms and conditions for repayment;
  • location chosen in terms of ecological zones and pastoral system, so that the intervention could be replicated in other areas;
  • land tenure reforms to guarantee access to pasture for poor and women-headed households and to include elaborated policies/regulations for detailed guidance for implementation; and
  • human resources development through appropriate training of management personnel and individual herders.

The design drew on IFAD experience and took into consideration the environmental/rangeland degradation risks and herders' preparedness for risk management.

The framework for IFAD assistance comprised:

  • Targeting the poorer segment of herding communities and the less advantaged area with special attention to herding households with few animals and those headed by women;
  • Promotion of income-generating activities (IGA), especially for poor households outside the livestock production system;
  • Restoring animal health services to reverse the rising trend in mortality rates of both adult and newborn stock; and
  • Provision of support for improving livestock marketing and removal of remaining government controls to redress the deteriorating terms of trade, facilitate access to essential inputs and broaden the market opportunities for poor households.

The project had four components:

  • Livestock distribution
  • Vegetable production
  • Project implementation and institutional support
  • Technical assistance (TA), studies and training

Project area. Arhangai aimag was chosen because it is almost entirely rural and heavily dependent on livestock production, with a high number of poor households, and communication with Ulaanbaatar. The target group was to consist of a mix of residents of aimag/sum centres and rural households. Initial beneficiaries were 6 600 poor and very poor households, of which 2 000 were to be herding households and very poor households, possibly with some animals. The proposed extension to a second aimag added 4 600 poor and very poor households, bringing the total to 12 000 households.

Livestock distribution. The main activity of the project was an in-kind livestock credit scheme, with beneficiaries who were full-time and experienced herders owning a maximum of 20 bod (traditional large ruminant unit) of animals. The maximum loan size was to be a minimum of five and a maximum of ten bod, with an average of seven bod.  The borrowers were to repay the equivalent of two mature offspring for each breeding animal with a repayment period of six years, including a one year grace period for sheep, and nine years including a three years grace period for cattle and yak.

Vegetable production and income generation. Credit was provided to households officially classified as poor and very poor to purchase input packages for vegetable gardens and IGA. The loan size for vegetable production was US$30 without provision of fencing and US$60 with fencing. The estimated number of beneficiaries was 1 800 households.

Implementation. At the national level, overall responsibility for project implementation, supervision, monitoring and liaison with IFAD and other government agencies rested with MOFA.  At the aimag level, a Project Implementation Unit (PIU) was responsible for coordination and implementation of the project activities. A Project Coordination Committee under the aimag Governor was responsible for the review of progress, approval of annual work plans and budgets and coordination.

Subsidiary Loan Agreement. The original Subsidiary Loan Agreement (SLA) signed between MOF, MOFA and Arhangai Aimag in 1997 stipulated that all funds drawn from the IFAD loan were repayable by the Aimag (contrary to the provision of the IFAD/GOM Loan Agreement whereby only funds required for the Livestock Distribution and Restocking Credit would be relent to the Aimag). The terms of the SLA were denominated in US dollars, thereby passing the foreign exchange risk to the Aimag. At the recommendation of the MTR in 1998, amended SLAs with Arhangai and Huvsgul were finally signed in May 2001.

Summary of implementation results   

Quantitative targets set by the Appraisal Report and revised by the MTR were exceeded for livestock restocking (72 per cent more animals were redistributed to 41 per cent more herders). A total of 3 494 households (1 433 in Arhangai and 2 061 in Huvsgul) received restocking loans. An average of 9.5 bods per household were distributed at an average cost of T 597 000 per household, against an appraisal cost of T 625 000.

Loan recovery in Arhangai was 100 per cent during the first two years, but declined over the lifespan of ARPAP, particularly following heavy losses of animals due to the dzuud (a freezing over of the snow in spring which makes it impossible for grazing animals to penetrate to the grass below) of 1999, 2000 and 2001, and the insufficient compensation paid to herders by the insurance scheme.  In Arhangai, only 28.6 per cent of those who took loans in 1997 are still repaying regularly, while in Huvsgul, 55.4 per cent of those beneficiaries who took loans in 1997 are still repaying on a regular basis.  Vegetable production activities included provision of loans to 4 276 beneficiaries through supply of seeds, tools, herbicides, etc. Repayment rates were 85 per cent in Arhangai and 71.8 per cent in Huvsgul.

Project performance

The initial project design and objectives were relevant both to GOM policy and to IFAD's mandate in that:

  • -the credit component was intended to target the poorest households, which in the Mongolian context were those with few or no livestock holdings;
  • credit in kind was intended to protect beneficiaries from price fluctuations in livestock prices in that if beneficiaries had to repay in cash and prices fell, they would be in a debt trap;
  • credit for female-headed households was emphasised in the design; and
  • research on issues of pasture sustainability, marketing and haymaking were all included in the design.

Relevance. IFAD sought to identify a model to replicate in other aimags and to assist poor herder households make full use of the potential advantages of a market-oriented economy, while ensuring adequate social and economic support services.  The project aimed also to reduce localised overgrazing around sum/aimag centres. The overall relevance rating of the project, in the light of the evolving economic and political conditions at the time of appraisal is 5.

Effectiveness.  An increase in the annual income and assets of poor and very poor herders was achieved only to a limited extent, since in Arhangai a majority of beneficiary households reported a decrease in herd size. In Huvsgul, most beneficiaries reported an increase. It should be noted that the project was operating within the context of the reforms introduced by the Asian Development Bank (ADB) Agriculture Sector Loan, whereby the role of the state to support herders in emergency situations like the dzuud of 1999, 2000 and 2001 was minimal, and emergency fodder reserves were extremely low.  However, it is also questionable whether the beneficiaries were ‘poor and very poor herders'; as the majority of beneficiaries had over 15 bods, which is close to a viable herd.

Regarding improved nutrition, reduced food expenditure and/or increased incomes of other poor and very poor households, the pattern in Arhangai and Huvsgul shows that production of vegetables and potatoes rose markedly during the period of implementation of the project, and then waned after project closure. Although the component appeared to be targeted correctly, the evidence suggests that it was not sustainable and has had little or no long-term impact on the incomes of poor households.

Concerning the developing of a replicable model of livestock distribution and vegetable production to implement in other aimags, the project approach did not take fully into account the IFAD experience with restocking projects over a long period.  These restocking projects have shown that this approach has not been very successful for a wide variety of reasons. Moreover, through discussions held by the evaluation team with donors in the country, it was evident that while restocking provided an important short-term safety net in rural Mongolia in the late 1990s, GOM and donors have now come to appreciate that it also raises serious concerns about long-term sustainability.  The vegetable production component performed higher than the restocking one; yet, a number of constraints to the effective implementation of the component were noted, such as: (i) limited marketing at soum level; (ii) shortage of good quality seeds; and (iii) water scarcity.  In light of the above considerations, the effectiveness of the project is rated 3.

Efficiency.  If the project were considered purely in terms of credit delivered against cost, the ARPAP was very efficient. Not only were more loans delivered than projected, the PIU also only spent 80 per cent of the budget, due to favorable foreign exchange terms. This was also possible due to the elimination of all TA and research components. In addition, animals for distribution were bought at fair prices on the market and inputs were delivered in a timely manner. In both cases, the beneficiaries interviewed had no complaints about the performance of the project. However, the issue is whether focusing on credit to the exclusion of all other types of public good (such as pasture research) is a desirable way to manage a project.  The efficiency of the project is rated 4.

Performance of IFAD.  IFAD underestimated the difficulties of operating in Mongolia.  Contextual difficulties were compounded by lack of country knowledge and country analysis capabilities.  The project design was based on IFAD experience from similar projects, and took into consideration the environmental/rangeland degradation risks and herders' preparedness for risk management.  There were, however, major design weaknesses, most notably: (a) inadequate awareness of the impact of dzuud; (b) an inadequate strategy for responding to other sources of animal loss; and (c) a failure to consider the viability of insurance. In addition, as some components were never implemented and as credit repayment has been operated in a manner not in line with IFAD standards, but unknown to the Fund until approaching time of project closure, IFAD's support to implementation was weak.  The rating is 3.

Performance of Government.  The GOM has continued to make poverty reduction a high priority and has generally been supportive of the project at the central government level. But its policies have not always had this effect on the ground. A lack of investment in rural areas, a concentration of mining as the major source of government income and a failure to consider the long-term consequences of cashmere production have increased inequality between the urban and rural areas.  The GOM appears to have complied with its agreements with IFAD on a financial level if not in terms of implementation. However, the signing of the SLA effectively undercut the intentions of the original loan agreement. Even when changes were agreed by GOM in principle (such as not passing on foreign exchange risks to the aimag governments, as under the original SLA) these were only slowly carried out in practice. The rating is 3.

Performance of the cooperating institution.  UNOPS carried out 11 supervision missions during the life of the project. Though the quality of supervision varied, the supervision missions identified and made recommendations on all relevant policy and operational issues, although with sometimes inconsistent and even contradictory concerns.  A number of important issues, such as revision of the SLA, implementation of support service activities and others, repeatedly highlighted in supervision reports, remained unresolved for long periods. The rating is 3.

Project impacts

The original target groups for ARPAP were ‘poor and very poor' households, split between herders and those with too few animals to build viable herds. This was to be achieved by setting limits to the bod holdings of potential beneficiary households. Households eligible for vegetable production and income-generating loans were those defined by the Local Development Fund as ‘poor and very poor'. The pressure to ensure repayment, a consequence of the terms of the SLA, encouraged the PIU to increase the level of livestock holdings required for eligibility.  The household survey, conducted at the time of the evaluation, shows that loans were given to 25 per cent of beneficiaries who already had more than 20 bods. If this were scaled up for all beneficiary households, it would suggest that 1 069 households were given loans to which they were not exactly entitled.  Regarding vegetable loan beneficiaries, no data on the economic status of vegetable loan beneficiaries is available in the PCR or from the PIU, but interviews conducted by the evaluation team all suggest that the beneficiaries were appropriate.

Physical and financial assetsIn Arhangai, 54 per cent of households that received animals from 1997 to 2003 reported a decrease in herd size, and only 46 per cent an increase in herd size by 2004. In Huvsgul, 30 per cent of households reported a decrease and 70 per cent an increase in herd size.  Although the main cause of these losses was the dzuud, its negative impact could have been reduced by the activities proposed at design (provision of hay-making machines, promotion of fodder production, etc.).  It is also true that GOM policy in this same period was to divest itself of centralised hay production and distribution, to cease collective wolf hunts and reduce government veterinary services. In such a policy climate, the losses were inevitable, although project design did not take sufficient account of this. Vegetable loans were more successful than livestock loans in improving access to material possessions, but in all cases, the percentage of households benefiting was never more than 50 per cent.  The rating on Physical and Financial Assets is 3.

Human assets.  The project was to implement skills' development training for vegetable producers, aimag and sum staff and for the PIU. In addition, herders were to be informed about repayment procedures, to be kept abreast of livestock prices and veterinary issues through regular radio programmes. The training of herders had limited impact on their livelihood, as training focused mainly on familiarizing beneficiaries with livestock loan procedures and selection criteria instead of skills training on livestock production and pasture management.  Also, although funds were allocated to mobilize consultants for technical advice, no expenditures were made for this purpose. Extension was rather weak: radio programmes were mostly used for disseminating information about the achievement of the project instead of TA for livestock and vegetable production.  The rating is 2.

Food security. If the project had achieved the projected results, the impact on food security would have been considerable. Poor herders would have viable herds, poor sum centre households would have both greater access to vegetables and a potential source of additional income to supplement the protein in their diets. The impact of ARPAP on food security overall has been confined to households where herd growth enabled loan repayment and on those where vegetable production is being sustained. The rating on this impact domain is 3.

The direct environmental impact of ARPAP has been minimal; the marked environmental changes since 1995 result from larger national policies and economic trends. The need to establish a range monitoring system capable of measuring range productivity and its changes over time and under increasing grazing pressures was not fully recognized by the PIU. Although these issues were highlighted in the MTR and in subsequent supervision reports by UNOPS, no action was taken until late in the life of the project. The rating is 2.

The impact on institutions and policies of ARPAP was low. The absence of monitoring, the lack of a practical research element and failure to make use of institutions such as the High Mountain Research Centre in Ikh Tamir (HMRC) gave it no authority to intervene in the policy arena. The problems of the repayment system have impacted most notably on female-headed households. All the households that lost their herds in the recurrent dzuud, are still being actively pursued to repay their debts.  The rating is 3.

Sustainability. Concerning restocking, it is arguable that it could have worked under the environmental conditions present in Mongolia. The frequency of dzuud was known prior to appraisal and it could reasonably be assumed that one or two would occur within the lifetime of the project. The safety nets provided in the Soviet era had all been eliminated by 1995, increasing the potential for animal losses on a greater scale. Given that the direction the Mongolian economy would take was difficult to predict, the project should have been treated as experimental and been more willing to respond rapidly to feedback from the ground with significant design changes. Regarding vegetable production, this was successful in some sums, but in others the growing season is too short for reliable yields. In addition, the increasing number of plagues of grasshoppers that are a consequence of pasture degradation are also attacking vegetable gardens, making the long-term sustainability of this intervention somewhat doubtful.  The rating for sustainability is 2.

The following table shows the ratings of the programme for performance, impact and overarching factors and performance of partners:

Ratings of the Arhangai Rural Poverty Alleviation Project

  Project evaluation ratings1
Programme performance  
Relevance 4
Effectiveness 3
Efficiency 4
Impact (overall) 3
Physical and financial assets 3
Human assets 2
Food security 3
Environment and natural resources 2
Institutions and policies 3
Overarching factors  
Sustainability 2
Gender 2
Innovation 2
Performance of partners  
IFAD 3
Co-operating Institution 3
Government 3

1/  IFAD uses a scale of 1 to 6, where 1 represents the lower score and 6 the highest.

Conclusions and recommendations

Overall, the impact of the project has been limited, and in a few cases, negative. Some households were assisted to develop a viable herd, but almost as many were saddled with serious long-term debt for which they are still being pursued. This burden has particularly fallen on female-headed households and is the cause of significant personal distress. The vegetable credit is probably the most successful element of the project; and although vegetable growing declined after the project closed, there has been a visible change of culture in sum centres and almost certainly improved nutrition for children. The project design, based on IFAD experience from similar projects, took into consideration the environmental/range land degradation risks and herders' preparedness for risk management. There were, however, major design weaknesses, notably inadequate awareness of the impact of dzuud, an inadequate strategy for responding to other sources of animal loss and a failure to consider the viability of insurance.

Recommendations

Debt relief and livestock loan repayment. A failure to consider the consequences of dzuuds serious enough to eliminate entire family livestock holdings left many households with insurmountable repayment problems. The financial burden on aimags created by the SLA has meant that the poorest households are still being harried for debts, and it is unrealistic to expect them to repay. This burden falls particularly on widows and single mothers because other beneficiaries with more resources have simply decamped.   IFAD and the GOM must thus find an institutional solution to resolve this issue.

Risk preparedness. By 2004, 40 per cent of project beneficiaries had less livestock than when they took the loan. The principal reason was the incidence of dzuud, a climatic phenomenon whose intervals of occurrence was already known prior to project appraisal.  Further dzuuds will occur and losses may well be greater as animal nutrition deteriorates due to degraded pastures.  Effective risk management must include the establishment of safety net mechanisms along with pasture management to avoid localized overgrazing.

Supervision. The project failed to execute many ‘soft' components of the project, such as socio-economic data collection and rangeland monitoring. Both UNOPS and IFAD should have acted to ensure compliance with the formulated recommendations; failure to do so has had adverse consequences for beneficiaries at the later stages of project implementation.  In future, IFAD and its Cooperating Institution need to ensure compliance with the project design and with recommendations of supervision missions.

Rangeland assessment. The rangelands in Arhangai and Huvsgul have undergone considerable damage since the implementation of the project in 1996, as evidenced by the major switch in the pastoral economy towards goat production. This occurred especially after the 2001 dzuud, an occurrence that has been verified by all households interviewed, and supported by the official statistics. Priority should be given to rangeland assessment and be actively and effectively pursued by all parties concerned in the ongoing IFAD Rural Poverty Reduction Programme in Mongolia.  In addition, IFAD should explore and capitalize further on knowledge concerning the technical aspects of extensive pastoralism in low-temperature regions.

LANGUAGES: English

Rural Development Project of the Mayan Communities in the Yucatan Peninsula (2006)

Mexico  
December 2006

Completion evaluation

Introduction

Background of the evaluation. The loan for the Rural Development Project of the Mayan Communities in the Yucatan Peninsula was approved by the Executive Board of the International Fund for Agricultural Development (IFAD) in December 1995. The loan contract was signed in July 1996 and took effect in November 1997. The project was scheduled to end on 31 December 2001, but two extensions have been granted, bringing the completion date to 31 December 2004.

Approach and methodology. The IFAD Office of Evaluation decided to send a mission to conduct a project completion evaluation. The mission1 visited the country from 22 November to 18 December 2004. Prior to its visit, the mission collected and analysed the documentation available at IFAD headquarters, and prepared evaluation instruments. Once in the country, the mission met with authorities from the executing agency and other institutions involved in IFAD cooperation, subsequently visiting the project areas, contacting organizations and interviewing directors and beneficiaries.

In undertaking the evaluation process, the mission followed the methodological framework adopted by the IFAD Office of Evaluation. IFAD's project evaluation methodology calls for close cooperation between the borrower, the project management and beneficiaries in jointly examining the progress and impact of the project.

On 13 December 2004, the mission conducted a workshop in the city of Mérida to present preliminary conclusions to officials from the executing agency, and on 16 December 2004, it held a final meeting in Mexico City with the parties involved to present the aide-mémoire summarizing the initial findings of the evaluation.

Main design features2

Project rationale and strategy. The project is consistent with the Mexican government's policy of promoting rural development through support for the reorganization of productive services3, encouraging autonomy and strong participation on the part of beneficiaries, which is also in line with the IFAD strategy that emerged from the Special Programming Mission in late 1992. Similarly, the provision of special supports for women and the most vulnerable social groups is consonant with the specific aims and priorities of IFAD and is of particular importance in an area that is one of the poorest in the country and that has a predominantly Mayan indigenous population.

Project area and target group. The project area comprises the three states that make up the Yucatan Peninsula: Campeche, Quintana Roo and Yucatan, which have a total of 64 municipalities and an indigenous population of approximately 670 000 people. The target group4 includes 51 100 families5.

Goals, objectives and components. The goal set for the project was to assist 10 000 families, including at least 30% women among the beneficiaries of the various planned activities. In addition, nine regional funds (RFs) were to be included within the sphere of action a similar number of indigenous coordinating centres (ICC); 520 productive projects were to be formulated, with financing provided for about 470; and 200 training events and about 45 studies or consultancies were to be carried out.

The general objectives established for the project centred around improving the income and living conditions of the target group, strengthening the development of local institutions to enable them to provide technical and financial support services for production and marketing, and establishing mechanisms that would allow for an equitable distribution of the benefits of the project.

For the achievement of the objectives, the design identified four main components: (i) investment financing; (ii) technology development services; (iii) promotion of demand for services; and (iv) the project executing unit. The gender perspective was considered a cross-cutting element essential to the implementation of all components.

The base cost of the project was of USD 16.22 million and the total cost (including physical and financial contingencies) was USD 17.2 million6, of which IFAD was to contribute USD 10.4 million and the borrower approximately USD 6.8 million. Of the total cost, 60% (USD 10.35 million) was for the productive investment financing component; 11.4% (USD 1.95 million) for the technology development component; 14.6% (USD 2.51 million) for the demand promotion component; and 13.7% (USD 2.36 million) for the execution component (project executing unit, plus programming and monitoring).

Implementation partners and arrangements. For its implementation, the project was to have a trust fund7, which would channel resources to the RFs, in addition to the project executing unit, located within the National Indigenous Institute [Instituto Nacional Indigenista (INI)] (now the Commission for the Development of Indigenous Communities [Comisión para el Desarrollo de los Pueblos Indígenas (CDI)]. Responsibility for the evaluation of project execution by the borrower was to be entrusted to qualified independent entities, such as universities or other specialized institutions.

The borrower's representative vis-à-vis IFAD is Nacional Financiera (NAFIN SCA). Other federal or regional public institutions could also be involved in the project, contributing additional funds or technical support. The United Nations Office for Project Services (UNOPS) was designated as the IFAD cooperating institution.

Major changes in policy and institutions during implementation. The project originated during a period of structural, institutional and economic change.8 Two significant events occurred in the course of its execution: a change of government, with the election of a president from the opposition, and a restructuring of the INI, which became the CDI, with a less centralist operating structure.

Design changes during implementation. The trust fund originally conceived as the mechanism for channelling funds to the RFs was replaced by administrative procedures of the executing institution9, and the RFs were given direct responsibility for administering the resources for investment financing. Federal funds channelled annually through the budget and earmarked for the project were intended to complement the funds from the IFAD loan.

The list of nine RFs to be assisted was expanded to 16, encompassing virtually all10 of the RFs existing on the peninsula.

The project fell behind schedule and did not really get under way until the second half of 1998, which led to two extensions of the project completion and loan closing dates.

The differentiation included in the original design in terms of assistance for the target population, which provided for the possibility of non-reimbursable financing with resources other than recovered funds, was modified11.

The PEU was not established in the normal way. Officials from the central office took over executive management of the project, while state representatives from the INI/CDI were responsible for coordinating execution in the states. Later on, a regional coordination office was created for the entire peninsula.

The technology assistance council (TAC), which was to have provided advice on the technical assistance activities of the project, ceased to function a few months after its creation12.

Summary implementation results

Financing for productive investments. For the period 1998-2004, the regional funds provided financing to about 420 groups (most of them mixed) for productive projects, extending credit to about 8 300 members, not all of whom met the requirements established by the project13 and most of whom continued producing and marketing their products individually.

Of the projects financed, 65% were livestock-related. The average amount of financing per project in 2004 was USD 7 412. The average amount per user was only around USD 662, but that amount was fairly significant in the small-scale economy of the borrower, generating self-employment and income opportunities. The percentage of women recipients of credit rose from 24% in 1998 to 55% in 2004, averaging 30% for the period as a whole, which was the established minimum.

Strengthening of the regional funds. The performance of the RFs assisted through the project showed positive changes: extension of the term of loans and increases in their amounts, greater participation by users, improvements in credit approval procedures14 and collections and in recovery rates and organizational aspects; management of accounting and automated information; and interest rates that were closer to inflation rates15. However, the RFs continue to be heavily dependent on resources from fiscal revenues.

The training provided, both for members of the boards of directors and for the group representatives, is still in a very early stage, and information on financial performance (FIS) is generally not accessible to decision-making entities in the manner in which it is presented, nor is it comprehensible to directors and representatives.

Technology development services. The project helped to improve the documentation accompanying applications for financing, and standards for the formulation of technically and economically viable projects were formulated. A review committee established in 2003 to assess the quality of proposed projects and approve applications helped to enhance the quality and transparency of the process.

The training programme has not had sufficient impact on the target population. Few of those interviewed showed that they had acquired sufficient additional knowledge and skills to enable them to carry on the activity generated by the project on their own. This deficiency was even more evident among groups of women and family groups of indigenous origin.

The RFs contracted for technical assistance directly, averaging 45 technical experts per year to assist all the groups financed by the project. Field observations showed small-scale improvements in basic technology, diversification of production and increased animal assets. Assistance for the sale and marketing of products was limited, and technical support for women was weak16. Members played an active and significant role in counterpart contributions to the loans of the RFs.

As of 2003, a total of 54 demonstration plots had been established on members' land in order to disseminate best practices and innovations in their communities, with varied results. In general, the impact of the demonstrations in terms of participants reached was limited.

The TAC that was to have advised on the technical assistance activities of the project functioned only for a few months. In the framework of its objective, some institutional or individual agreements for cooperation and assistance were reached, which remain in effect, although the advisory services and studies provided were insufficient to ensure the expected level of specialized advice.

The productive activities financed, most of which were very limited in scale, had little or no effect on environmental conditions17. Moreover, the project resulted in some growth in organic farming.

Promotion of demand. According to information from the executing agency, fewer than 20% of the RF member groups have been able to reach the consolidation stage, as they have not achieved financial autonomy and self-management. Most are still in a phase of formation or strengthening, and their membership diminished when the first financing was obtained. Regional meetings and training, together with systematization of experiences and other mechanisms, have contributed to closer ties between the groups and the RFs.

The capacity of young people (members or associates of these groups) to serve as links between the instructor and the group members was not exploited, nor was use made of personnel available in communities who had training in promotion techniques.

The assistance for indigenous school residences and the use of the radio in the Mayan language were limited in time and scope18, and there was little distribution of instructional support materials for training, promotion and dissemination, designed by those responsible for "communication".

The percentage of female credit users increased significantly, rising from 23.65% in 1998 to 55.15% in 2004 and the presence of women involved in leadership of the RFs also grew. However, there is still a great deal of unmet demand among groups composed primarily of women19, despite their good performance as borrowers.20

Project management and coordination. One of the advantages of the organizational model adopted for project execution was institutionalization of the measures introduced to strengthen the performance of the RFs, as well as of measures aimed at improving the quality and execution of the projects formulated. Some progress was observed in decentralization and coordination of decision-making and activities.

Monitoring and evaluation of execution. Progress was noted in the implementation of the system for monitoring project execution, but there continue to be problems with timely and ready availability of the data collected and with use of the information for feedback.

Performance of the project

Relevance of objectives. The objective of developing a system for financing productive activities through regional solidarity funds (RSF) or regional indigenous funds (RIF) and strengthening these institutions remains very valid, given the need to ensure access to financing for excluded rural populations.21

The promotion and strengthening of community-based organizations and of their affiliation with the RFs, coupled with the dissemination of information in the Mayan language about the RFs and their services, proved to be effective ways of encouraging the active participation of the target population and channelling their demand for financing.

The goal of reducing inequity in the project area is in line both with current social demands and with the official policies and the specific aims and priorities of IFAD.

Effectiveness. The project has shown progress in increasing the value of family work and in opening up new productive options. In successful cases, it led to expansion and improvement of productive activity, increased the incomes of beneficiary families and enabled families to build some capital and reduce their vulnerability.

Project support contributed to the strengthening of the RFs; however, the local capacity of these institutions is still insufficiently developed to enable them to achieve participatory and autonomous management.22

A significant number of producer groups and community-based organizations have become members of the RFs, in general family-based23 groups; however, they frequently do not feel real ownership of the RFs and have not, in practice, achieved the capacity to work as a cohesive group.

During project execution, mechanisms were adopted to reduce the existing inequities in the communities and groups of members. Nevertheless, the beneficiary selection criteria established for this project were not strictly applied.24

Efficiency. As the project neared completion, virtually all of the external resources allocated and most of the counterpart funds had been disbursed, and the physical goals had been met to a satisfactory degree. However, despite the advances mentioned above, there were some deficiencies in execution, including high operating costs of the RFs and low recovery of loans; training activities that were not sufficiently structured and differentiated; technical assistance that lacked continuity and was not always appropriate; insufficient promotion; and little use of opportunities for dissemination and communication. In addition, the use of monitoring indicators for purposes of project management is incipient, and expenditure on instruments for the coordination of project activities in fact did little to improve coordination.

Impact on Rural Poverty

Impact on physical and financial assets. Members have made small investments for productive purposes (acquisition of animals, mainly). Any surpluses have not generally been used for household equipment or home improvement. Rating25 for impact on physical assets: 4.

Impact on human assets. The training activities and technical assistance provided under the project, with the weaknesses noted above, helped to strengthen human capital to some extent in the target population for the project. Rating for impact on human assets: 4.

Impact on social capital and empowerment. Owing to the deficiencies in the RFs and their member organizations and groups, the project contributed only in a limited manner to the formation of social capital. Rating for impact on social capital: 4.

Impact on food security. The mission detected some improvement in productivity (in physical terms) and increases in production, although they were not always accompanied by increases in the profitability of the activity or the income of the producer.26 The acquisition of animals, the increases in production and, in successful cases, the achievement of surpluses helped to improve the food security of members to some extent. Rating for impact on food security: 3.

Impact on the environment and communal resource base. The productive activities financed were complementary and small in scale. Additionally, organic production and processes with low environmental impact were promoted. Impact rating: 4.

Impact on institutions, policies and the regulatory framework. The project has helped to bring about institutional improvements in the CDI and other development programmes27. Impact rating: 5.

Impact on gender. The project has helped to make visible the contribution of women to productive activities and ensured their presence in groups and organizations and in the RFs. Women's access to the benefits of the project was significant in terms of access to credit, but much still remains to be done in order to achieve more effective participation and representation of women. Impact rating: 4.

Sustainability. Despite the contributions of the project, the RFs involved are unlikely to be sustainable: the loan recovery rate is still low and income from interest on loans and other sources is not sufficient to offset high operating costs. In addition, continuity of the technical assistance after the project ends is doubtful. Rating for project sustainability: 2.

Innovation and replicability/scaling-up. The measures implemented by the Yucatan Peninsula Project to strengthen and improve the performance of the RFs28 have been institutionalized by the executing entity, and their application has been promoted in other regions. Impact rating: 5.

Other poverty impact. The mission did not note any impacts on poverty other than those reported above.

Overall impact assessment. The project has had some impact on the rural agriculture context of the Yucatan peninsula, helping the traditionally excluded and vulnerable population to begin to access productive options that could pave the way towards future undertakings. Overall impact rating: 4.

Performance of partners

Performance of IFAD. IFAD has maintained constructive relationships with the national counterparts responsible for administration and application of the loan funds. Users' appraisal of IFAD's participation is generally favourable. There were some delays in processing disbursements in the course of project implementation. IFAD carried out supervision and support visits to the country and the project, which were complemented by the activities of the cooperating institution. Rating for IFAD performance: 5.

Performance of the cooperating institution. The performance of UNOPS was consistent with its terms of participation. The information and suggestions presented in its reports have provided useful guidance, although some of its reiterated observations were never applied in the execution of the project. Rating for UNOPS performance: 5.

Government and its agencies (including project management). The counterpart financing to the IFAD loan, particularly in the beginning, was reduced significantly, which affected project performance. In addition, there were delays in releasing the resources allocated. Overall performance rating: 4.

The performance of the executing agency (CDI) showed steady improvement. In general, it was able to meet its objectives and carry out its managerial functions. However, excessive interference in the RFs was noted, as were some weaknesses or difficulties in the coordination of activities, in the training approach and strategy, in the selection of extension agents, in the dissemination of information and in the performance of the monitoring system. Rating for performance of the executing agency (CDI): 4.

Performance of non-governmental and community-based organizations. The RFs increased their assets and made some headway towards being able to operate as self-sustaining and participatory financial intermediation entities. However, they remain highly dependent on fiscal resources, and they did not succeed in bringing about any significant increase in the availability of financing for the target population.29 Performance rating: 4.

The fact that families comprised many of the producer organizations and groups lends them a degree of permanency; but these groups lack capacity for self-management, they are not really integrated into the RFs, and their members continue to approach their work basically as individuals.

Overall assessment and conclusions

At its completion, the project, which was extended twice30, shows a performance that can be considered satisfactory from the standpoint of achievement of physical and financial goals.

The project's impact in improving the income and living conditions of the beneficiary population was modest; but for families of limited means, it afforded a valid opportunity for getting involved in new productive activities and strengthening existing ones in a context of greater responsibility and self-determination31. Although it was not possible to ascertain the exact extent of the change in production and/or productivity32, observation in the field revealed improvements in basic technology, diversification of production and an increase in animal assets33.

The project helped to increase the number of and strengthen the RFs, although it was not possible to achieve a level of capacity development that would enable them to function in a truly autonomous and participatory manner.

The financial performance indicators of the RFs improved steadily over time.34 The approaches and mechanisms introduced by the project, in terms of financial, administrative and accounting management35 and support for productive investments, have made an important contribution to the implementation of development programmes and have been institutionalized by the CDI and adopted in other regions and programmes.

Although the boards of directors of the regional funds prepare annual programmes of activities, the mission did not find any cases of FRs that prepared an annual budget or any other estimate of expected income and expenditures. In addition, the accounting information on the performance of the loan portfolios of the RFs is, for the most part, cumulative.

The project has helped to bring a great number of producer groups and community-based organizations into the RFs, most of them family-based groups36 motivated by access to financing. In practice, however, most still lack to the capacity to manage financing, obtain technical support for production, or work and market their products collectively.37

The project contributed effectively to the improvement of applications for financing and to the development of technically and economically viable productive projects38. With regard to technical assistance for production, the mission noted problems with continuity and lack of specialization and suitability of technicians and trainers.

The fact that the project explicitly set a minimum percentage for women's participation in its activities and suggested ways of increasing women's leadership and direct involvement in the decision-making of production groups and RFs has yielded positive results. In general, the most recent RFs have shown improvements in their management, with more participation by women than older funds.

The training programme objectives were not fully realized among the target population, particularly women's groups.39 The communication subcomponent yielded some positive results. The use of the radio in the Mayan language to publicize the project was effective and showed high potential. Similarly, the indigenous school residences proved to be effective vehicles with high potential for transferring innovative and best practices.

With respect to the environment, the project activities and approaches have, in the vast majority of cases, been respectful of the traditions and values of the predominant cultures and of natural resources and the environment generally, providing support for organic and balanced production processes.40

The efforts made and the advances achieved by the executing agency were insufficient to overcome weaknesses in the coordination of activities, the systematization of experiences and the compilation and organization of relevant data, as well as the dissemination of related information.

IFAD's involvement was decisive in making the loan operation a reality. The cooperating institution's role was consistent with its terms of reference for project supervision.

Insights and recommendations

Insights

An important strength of the regional funds assisted under the project is that they have brought together organizations comprising members of varying political positions, religious beliefs, age and sex, although the organizations themselves have all been relatively homogeneous from a cultural and socioeconomic standpoint.

The absence of a clear definition of the nature of the RFs (financial, social or financial-social) makes it difficult to plan strategic actions that would help ensure their sustainability over time.

Proper selection and training of technical advisors, as well as conditions of job stability and incentives, are crucial conditions for achieving quality and efficiency in the provision of technical assistance services.

The use of radio broadcasts and communication in indigenous languages offers great potential for publicizing projects and for informing and training the target population. This resource should be utilized efficiently.

The fact that the project executing agency was the institutional structure of the CDI itself, reinforced strategically, may have had an impact on the improvements and changes in approaches and priorities introduced by the project, which were progressively institutionalized.

Recommendations

Decentralization of implementation. Take concrete action to ensure that organizations and communities of the target population are given increasing responsibility for decision-making about project implementation and resource administration.

Regularity in resource allocation. Take steps to ensure that project budget funds flow as scheduled.

Definition of RFs. Clearly establish the financial and social nature of the RFs and plan their strategic actions accordingly.

Institutional development of RFs: Protect RFs from political contingencies, strengthen their diverse nature as representatives of low-income rural populations, promote their involvement in participating communities and emphasize capacity-building for and participation by their directors.

Training for RF directors. Inform and train RF boards of directors adequately with respect to legal and institutional instruments relating to the institution's activities and specific circumstances, as well as basic aspects of managing and interpreting accounting records and useful instruments for that purpose.41

Recovery of subloans. Continue to stress the need to improve loan applications and productive projects and the need for transparency and rigour in approval processes. Emphasize the responsibility of borrowers to repay loans, and assess the real possibilities for recovering small loans to populations with minimal assets and productive resources.

Diversification of financial sources. Arrange for the use of the various sources of financing available from the Mexican government to alleviate poverty and promote rural development and stimulate autonomous access by the RFs to those resources.

Quality of the technical assistance model. Conduct an in-depth review of the technical assistance model adopted in order to make future service more effective and more efficient. Develop an intervention strategy diversified by category of family beneficiary according to sex and age. Consider, for the medium and long term and once productive units are operating efficiently and producers have raised their incomes, including part of the cost of technical assistance in the loan.

Specialized assistance. Ensure the specialized advisory assistance needed by the technical assistance service for productive projects financed by the RFs.

Gender perspective Include an explicit gender perspective in designing projects and in dealing with social and organizational issues.

Support for demonstration plots in residences. Step up financial and technical support for activities such as those included in the indigenous school residences subcomponent of this project.

Exchange and systematization of experiences. Strengthen the promotion of and support for events to exchange and systematize the experiences of various projects and institutions.42

Training of trainers. Training of trainers43 should receive the full support of the CDI in order to empower the population and lend sustainability to the improvements introduced. Promote efficient and effective strategies and mechanisms as a complement to technical assistance in order to ensure social and generational equity44, as well as cohesion of groups.

Producers' demonstration plots. Encourage "demonstration plots" on land owned by members for applied research and testing of improved techniques, ensuring that they fulfil their intended demonstration and dissemination function.

Strengthening of organizations. Promote the idea that the objective of creating a group should extend beyond a simple short-term purpose, need or interest. Help strengthen the internal organization of groups and communities for decision-making purposes, and step up participation by current and potential beneficiaries in deliberative and decision-making assemblies and bodies. Also promote the concept of ownership among FR member groups so that they understand that the FRs belong to them.

Information and diffusion. Motivate and support the RFs in the intensive dissemination of their programmes, regulations and requirements.

Institutional experience with implementation. Conduct an in-depth review of experience with the direct implementation of projects by a pre-existing national or regional institution having been strengthened to that end.45

Replicability of RFs. Consider the possibility that the RF model, with adjustments and refinements as needed, could be replicated in order to strengthen organizational, economic and productive aspects of rural and indigenous communities in Mexico.

Coordination of project activities. Promote the incorporation of mechanisms to strengthen the coordination of project activities.

Monitoring of implementation. Require and provide the necessary support to enable executing institutions to develop and maintain up-to-date project indicators and instruments46 that will facilitate monitoring and timely corrective action during implementation.

Following recommendations. Take steps to ensure that recommendations arising from supervision visits and evaluations are in fact analysed and, where appropriate and relevant, applied by the executing institution.


1/ Consisting of Mr Raúl Alegrett, Mission Leader and expert in project management and institutional development; Ms Omaira Lozano, expert in microfinance institutions; and Ms Myra Speelmans, gender and evaluation expert.
2/ This information refers mainly to the original project design (developed during the appraisal phase). As is noted at the end of this section, several modifications were introduced in the course of project execution.
3/ Basically, access to credit and technical assistance.
4/ The appraisal document specified that to be included in the target group families had to meet the following criteria: (i) residents of a rural area, (ii) indigenous households, (iii) production units of around 5.0 ha. and income falling below the poverty line, which at the time was USD 400 per capita a year.
5/ Of the total number, 33 836 families lived in the state of Yucatán, 8 184 in Campeche and 9 080 in Quintana Roo (1995).
6/ The amounts for the project components are listed in the table entitled "Financing plan by component" in the December 1995 report and recommendations of the Chairman of the Executive Board of IFAD.
7/ The trust fund was seen as an appropriate institutional mechanism for providing financing to the target group and strengthening the institutional sustainability of the RFs and the farmers' organizations associated with them, making it possible to avoid their being subject to the control of public agencies, facilitating private investment and reorienting the role of public institutions towards strengthening of basic infrastructure.
8/ Particularly in late 1994, Mexico underwent a grave economic crisis.
9/ The CDI is required to negotiate its budget funds with the Secretariat of Finance and Public Credit [Secretaría de Hacienda y Crédito Público (SHCP)].
10/ One additional recently created fund (Escárcegas) received assistance only in the form of  training.
11/ With the acceptance of supervision.
12/ However, its objectives were borne in mind and several specific agreements were established with accredited institutions.
13/ The criteria for the target population of the project set limits on farm size and family income.
14/ The adoption of technical criteria (technical, financial, environmental and social feasibility) for the approval of credit applications, coupled with the implementation of qualification committees in 2003, improved the prospects for loan recovery.
15/ The interest rates currently being applied by the funds range from 6 to 12%, which, compared to the interannual rate of inflation for 2004 (4.5%) seem adequate; however, operating costs (travel, food and lodging, collection activities, etc.) amounted to considerably more than that percentage. There are some regional funds which, based on the financial indicators applied, show an expenditure/income ratio of 369%, with a strong tendency towards loss of capital and, potentially, failure.
16/ This was no doubt because the project did not have specialized technical support for these target groups.
17/ Mostly backyard production, with little or no increase in the areas devoted to crop or livestock-farming.
18/ Although more than 90 school residences were involved over the life of the project, in most cases the assistance was not sustained.  
19/ Most of the women interviewed indicated that their requests for financing: (i) had not been answered promptly; (ii) had been channelled towards other areas of production; (iii) did not have the technical support needed; (iv) were not accompanied by training suited to their needs and opportunities.
20/ In addition to being good "payers", women in groups engaged in "traditional activities" were able to generate family savings (generally in the form of farm animals), earn a small income from performing their day-to-day tasks and, with the second loan, acquire some goods or equipment for the performance of their productive activities.
21/ The project area is one of the poorest in the country, with a large proportion of marginalized population. Apart from simple subsidies, the RFs are about the only option available to this population for accessing financing.
22/ For example, although the quantitative achievements are satisfactory in terms of number of beneficiaries served and number of planned activities and events carried out, the same is not true of the number of well-trained managers and members, knowledge on the part of the population of the characteristics of the RFs and the project, the possibilities for membership and the responsibilities of members.
23/ Extended families or several nuclear families from the same extended family, together with nearby neighbours.
24/ The mission found several instances of RF participants whose income levels and land holdings were significantly larger than the established parameters and noted that in practice these groups with higher incomes and social status obtained better and bigger results than the IFAD target population.
25/ Based on the rating scale of 1 (negligible) to 6 (very high) used by IFAD.
26/ Lack of availability of accurate data on basic variables makes this measurement difficult. The project monitoring and evaluation office expects to have this information ready by the administrative closing date.
27/ Operational innovations, development and implementation of operating guidelines, manuals, norms and procedures.
28/ Introduction of technical and managerial elements, improved efficiency in their management, increased transparency, better technical quality in the design and execution of projects financed by the RIFs as a basis for supporting beneficiaries' production through financing and improving project execution through technical assistance.
29/ The RFs have been operating with non-financial resources (non-reimbursable funds from fiscal revenues), with little or no diversification of income, no capture of savings, low interest rates, low loan recovery rates and relatively high operating costs.   
30/ Institutional changes, budget cuts and decentralization of  the operational units of the executing agency (INI/CDI) short after the start of project implementation hindered disbursement of the loan funds.
31/ The activities of the RFs in the framework of the project have helped to raise the value of family labour, generating increased family earnings and, in general, increasing opportunities for productive work among members. Moreover, for the project's target population, the RFs are the only credit mechanism currently available, except for subsidy-based programmes.
32/ The project monitoring and evaluation unit expects to have this information ready by the administrative closing date.
33/ Any funds remaining from productive projects have generally been used for investment in animals, which has reduced the vulnerability (risk exposure) of member families at least to a small degree and has enabled them to address unexpected needs.
34/ The recovery rates for financing provided in the last three or four years (since 2000) have been much better than in earlier years. The RFs have also exhibited improvements in their boards of directors and greater concern with becoming self-sustainable.
35/ Including operational guidelines, manuals, norms and procedures.
36/ Extended families or several nuclear families from the same extended family, together with nearby neighbours.
37/ Many of these deficiencies may stem from the lack of strategic systems of support for the community and the groups that comprise it, as well as the absence of technical personnel with training in social areas (organization, training-communication, anthropology and gender).
38/ The project review committee established in 2003 to assess the quality of proposed projects and approve applications helped improve loan recoveries in 2004.
39/ Very few of those interviewed demonstrated that they had acquired sufficient additional knowledge and skills to enable them to carry out their new activities independently.
40/ However, the very limited scale of the interventions financed, especially in terms of farm size, lessens their potential environmental impact.
41/ This training should include assembly delegates and/or their children.
42/ Both among RFs and among technical assistants from different regions and member organizations.
43/ Both technicians and producers.
44/ In order to take advantage of opportunities (experience and knowledge of elders, capability and energy of youth) and reduce the inequities that exist within participating families (limited access to goods and services, excessive workloads for women).
45/ Important advantages include the institutionalization of the programmes, measures or instruments introduced, possible economies of scale and improved prospects for sustainability of the actions undertaken. Among the negative aspects are the risk of diluting efforts and attention, diversion of some project resources to other programmes, change in the nature of programmes and greater influence of partisan politics.
46/ Input of information requires constant effort to ensure consistent and complementary action between the state delegations and the RFs so that the information is readily available in a timely fashion. Priority should be given to using data to provide feedback to operating units and RFs in the form of reports addressing their specific needs.

 

LANGUAGES: English, Spanish

Fouta Djallon Agricultural Rehabilitation and Local Development Programme (PRAADEL) (2005)

Guinea  
December 2006

Interim Evaluation1

The purpose of the interim evaluation mission (IEM) – required under IFAD policy – was to determine whether the programme results and impact warrant pursuing the project and, if so, to consider whether the original approach ought to be modified or changed and formulate recommendations accordingly.

The mission arrived at Conakry on 21 November 2004, and met with the appropriate ministries and departments and the PRAADEL team. The mission worked in the field from 24 November to 9 December 2004. A feedback meeting took place on 15 December at the Ministry of Agriculture and Animal Husbandry, with participation of PRAADEL and other partners.

The mission evaluated the following: (1) the programme's impact on living conditions for beneficiaries (smallholders, women and young people); (2) the appropriateness and effectiveness of actions undertaken relative to the needs of the rural populations and programme objectives; and (3) performance by the project partners. Specific themes were also analysed, including the Gestion des terroirs approach, synergy with other projects and methodology for targeting beneficiaries.

PRAADEL covers 21 500 km2 and a rural population of 460 000. The objective was to reach some 300 villages, and specifically 22 000 smallholdings (maximum 1 ha) and 145 000 people. Priority groups included small-scale producers; women (including heads of household), who were to comprise at least half the beneficiaries; and young people, in particular unemployed young people.

Overall objectives were stated as follows: to improve living conditions for the target groups in terms of income, food security and health; to check gradually the degradation of natural resources and improve the sustainability of agriculture; and to promote local development based on grass-roots organizations representing the target groups.

Immediate objectives are: (i) increasing productivity, production and marketing of crop and livestock output; (ii) fostering local participatory management of natural resources under the Gestion des terroirs approach; (iii) strengthening grass-roots organizations and promoting effective participation of target groups in management and implementation of activities and improving the status of women through increased participation in programme activities; (iv) improving the accessibility of villages (rural tracks) and expanding drinking water supply (improving nutritional and health conditions); (v) putting in place a viable system of beneficiary-managed local financial services.

In order to achieve these objectives, PRAADEL was structured in three components: (i) Gestion des terroirs and local development; (ii) development of financial services associations (ASFs); (iii) development of rural infrastructure (rural tracks, management of waterholes and valley bottoms); and a programme coordination unit (UCP). Under the Gestion des terroirs approach, activity planning and implementation is organized around agro-ecological areas (natural resource areas). Moreover, under its faire-faire strategy, the project plans and coordinates but leaves implementation up to the stakeholders concerned (beneficiaries, public services, enterprises, peasant organizations and NGOs) with whom it has a contractual relationship. 

Evaluation Findings

The programme objectives remain valid and consistent with the rural development policies of both the Guinean Government and IFAD, and with the 2001 Poverty Reduction Strategy Paper (PRSP). The latter currently serves as a framework for all activity in relation to poverty reduction, although it has not yet been applied at the sector level.

At the time of the mission, more than five years after programme start-up, the IFAD loan has been 64% disbursed, but results are far from achieving the fundamental objectives. PRAADEL start-up was delayed, and processing of the IFAD loan by the Guinean Government has been irregular. However, this does not justify the discrepancies in implementation and poor results in the field: 6 sub-prefectures (Rural Development Councils or CRDs) of the 40 planned have been covered, 133 villages of 347 have participated, and Gestion des terroirs has been applied to 15 local units (terroirs) out of 25. The Local Investment Fund (FIL) has not been set up, promotion of production chains and strengthening of women's groups have been very weak, and 10 ASFs have been set up of the 21 planned for. In addition, 140 km of rural tracks have been established of the anticipated 400 km, 12 bridges and fords built of 70, and 20 waterholes established of 200. Finally, just 38 ha of valley bottoms have been treated of the 200 ha projected.

Analysis of Programme Performance

The implementation of PRAADEL has suffered a considerable delay. Pilot phase duration has been cited variously by different sources as two to four years, and the PRAADEL team's understanding is that the project is still in the pilot phase. Moreover, consistency between objectives and expected results as presented in the text and in the logical framework of the appraisal report was never established, nor was the original logical framework revised accordingly. Neither the UCP nor the United Nations Office for Project Services (UNOPS) nor IFAD took part at the time of start-up to ensure harmony between the content of the text and the logical framework, and/or later to arrive at a more realistic version of the projections as a whole.

The Gestion des terroirs approach requires that staff first of all, and then beneficiaries, master the methodology and the tools for long-term participatory implementation. Therefore, a certain amount of time is needed to be allowed for information and awareness-raising campaigns, for preparation of village development plans, and for establishing collaboration with CRDs in order to render the approach operational. Nevertheless, taking into account IFAD's presence in the same geographical area since 1990, these conditions ought to have been in evidence at the time of programme design and built into the planning. Accordingly, it is clear either that planning has not been optimal, or that the approach has been less efficient than anticipated.

Programme impact

The programme's achievement rate is low, and its impact can be observed only in cases of large-scale intervention. The programme has had an overall impact on the three beneficiary groups, but has made no significant changes in income-generating activities for women and young people.

Impact on material and financial resources. The target groups have seen an impact on a one-off basis following the planting of living hedges and fences and treatment of valley bottoms. Impact is negligible with respect to crop and livestock equipment. The construction of rural tracks has improved transport conditions for people and goods, as reflected in the construction of village buildings and the opening of markets.

Impact on human resources. Water points improvement has enhanced living conditions in 25 villages and reduced workload, in particular among women and children. The creation of health care stations has improved access to health care and birthing services for some 2 000 people. The construction of primary schools has raised the percentage of children (including girls) in school. The literacy level has risen in some locations.

Impact on social capital and collective capacities. The application of the Gestion des terroirs approach, planning (local development plans – PDLs) and institutional development (land development committees – CDTs) has contributed positively to improving social cohesion and self-help. The creation of ASFs and their committees has improved financial management in the villages concerned. For local populations, introduction of the Gestion des terroirs approach has strengthened awareness of the need for active participation in local development and stronger relationships with the private sector. The implementation of the Gestion des terroirs approach has also improved women's participation in public affairs, although most are still timid about taking part in discussions and decision-making. 

There has been a positive impact in the area of food security and household economies in cases where valley bottoms have been treated successfully. Access to financial services has enabled (married) women to position themselves more favourably in small livestock trade.

A positive impact on the environment and natural resources is evident in a greater willingness to respect the recommended frequency for burning to clear land, limit forest fires and thus protect greenery. The establishment of nurseries has facilitated reforestation and protection of certain locations, in particular headwaters. No overall reduction in the risk of environmental degradation has been observed.

Impact on institutions, policy and the regulatory framework is taken into account through the creation of 15 CDTs and their participation in implementing national policy. The existence of ASFs has helped mobilize savings among villagers. Some 2 000 member shareholders are active in the ASFs, and knowledge about small rural credit systems has improved.

Impact on women's status and gender issues. In this area, impact relates to the positive effects of infrastructure and financial services, the establishment of CDTs and the Gestion des terroirs approach as a whole. Impact in the form of strengthening women's production, women's groups and economic interest groups (GIE) is negligible. 

Sustainability of impact. With respect to the Gestion des terroirs approach, the building of infrastructure and the ASFs, there is a positive impact, but it will need to be consolidated to ensure sustainability. Impact in the areas of agriculture and the environment is very limited.

Validated innovations and potential for replication. The Gestion des terroirs approach is new in Guinea and will need to be supported if collaboration is to be assured among CDTs, CRDs, prefectures and the region. Potential replicability exists for CDTs and PDLs. The ASFs are also an innovation in Guinea and remain in a testing phase. In their current form, they can answer well the demands of small business but not the needs of agricultural development.

Other impact on poverty – targeting of beneficiaries. The direct beneficiaries of PRAADEL (small-scale producers, women and young people) are involved through the Gestion des terroirs approach, the existence of committees and village development plans (CDTs, PDTs) and tools to identify development needs. The programme has reached 133 villages, 2 583 families and, in terms of individuals, 8 500 men and 11 333 women. Impact in the form of changes in the economic situation of women and young people is negligible. Capacity among CDTs has not yet achieved a level that would enable them to undertake a more in-depth identification of needs for training, small business creation, organization of transportation, product processing, etc.

Overall assessment of impact and results compared to expectations. Considering that the initial objectives and projections remain unchanged, the overall impact of PRAADEL's activities is very limited. Globally, agricultural production has not improved, income-generating activities have not changed for women and young people, the exodus of the young remains undiminished and there is no change in the ageing of labour in villages.

Assessment of Performance by Partners

The mission has not identified any major deficiencies on the part of IFAD as the lead financial institution. However, it would appear that preparation of the IFAD programme was overly optimistic in terms of quantitative projections.

No major problems have been identified with respect to UNOPS as a cooperating institution. At times, agreement on payment authorizations took longer than planned, owing to difficult communication conditions in Guinea. Given the duration of PRAADEL and the considerable delay in achieving results under the programme, the mission regrets that there was no intervention on the framework of the partnership, which might have led to a review of expected results and the logical framework. 

The Government of Guinea. There have been considerable delays in the transfer of counterpart funds, leading to interruptions in PRAADEL operations.

With respect to the programme coordination unit (UCP), the mission made the following observations: public involvement in PRAADEL implementation has been very limited, and there have been cases where contracts with partners have not been respected, as well as irregular payment delays. The role of PRAADEL within the framework of regional coordination is seen as weak by a number of partners. The UCP has evolved in isolation, without developing any real synergy with other projects or programmes operating in the area. There is no training programme to build capacity among staff. Of the 34 people (14 of them professionals) employed by the programme, there is only one woman.

Also, the lack of consistency between anticipated results as stated in the text and the logical framework included in the appraisal report and its update indicate a lack of interest on the part of the UCP, UNOPS and IFAD in the baseline data and documents as instruments for programme management.

The majority of infratructural works (under the supervision of control missions) have been performed well (over 80%) and in accordance with the state of the art. Delays in works execution have never been attributable to villagers, who have always complied with the terms of contracts by contributing construction materials and supporting entrepreneurs and their employees. Among NGOs and consulting firms which are partners, there are some shortcomings in contract preparation, or the terms of reference for operators are not defined with precision.

Conclusions and Recommendations

In view of the relevance of the objectives to the situation in the programme area, as well as their alignment with the Government's agricultural policy, and in view of the pressing needs prevailing in the project area owing in part to the low level of achievement by PRAADEL, the mission feels that extending activities in Fouta Djallon, supported by IFAD financing, is justifiable should the Government of Guinea and IFAD so desire. Nevertheless, it is clear that certain elements of the current programme need to bear close scrutiny given the low achievement rate and changes attributable to progress on decentralization.

First of all, the Gestion des terroirs approach will be the main issue for review, in particular its usefulness in the current policy and administrative context. This approach is warranted above all with respect to natural resource management, where organization based on agro-forestry-ecological considerations is in order. Nevertheless, in view of the diversity of PRAADEL activities, one might well wonder whether this continues to be the same project and whether, as many kinds of public activities and services are involved, it might not be more relevant to plan and implement them on the basis of the corresponding administrative units in charge of such activities and services rather than the territorial unit or terroir. More specifically, it would seem reasonable to base the planning of valley bottom development on the terroir. However, with respect to roads, water supply, schools and health centers, such an approach appears illogical. An in-depth analysis of the nature of future activities and the most appropriate modalities for their planning and implementation should therefore be undertaken. Replicability and sustainability should be part of this analysis.

The local population has expressed appreciation for the activities carried out under the current programme, not surprisingly, since they have improved living conditions, and the mobilization of the population around such activities has had a positive effect on political and social capital creation. The question is whether these positive effects can be safeguarded or created if a different implementation approach is selected in the future. Opportunities for cooperation should be explored between this programme and the Village Communities Support Project (PACV) that provides support for the decentralization process, as well as with other projects operating in the area.

The beneficiaries of PRAADEL do not participate as employers in the awarding or payment of contracts for works and goods, unlike the procedures governing other projects (e.g. PACV).

Recommendations: (i) an in-depth study of the relevance of the Gestion des terroirs approach in view of decentralization and geographical boundaries; (ii) a comparative study of intervention mechanisms applied by PRAADEL and PACV; (iii) an analysis of the possibility of a gradual shift from the faire-faire approach to a faire-avec approach to give greater accountability to local communities; (iv) the programme should develop true synergies with other projects and programmes in the area; (v) in view of the priority accorded under the programme to women and young people as beneficiaries, the mission believes that including women among lead personnel would be helpful.

With respect to the UCP, administrative processing has been very slow, leading to frequent delays in activities considered important and urgent by populations. Additional constraints are: (i) a lack of cohesion between component leaders and the UCP; (ii) low morale among staff; and (iii) a lack of regular staff training.

Recommendations: (i) Improve the dissemination of information and build management capacity; (ii) apply a recruitment policy based on professional criteria as well as gender; (iii) draw up an action plan favouring collaboration with other regional development actors. 

The Gestion des terroirs component has been slow, and it would have been relevant to separate it from agricultural development and natural resource management. Planning methods did not give sufficient attention to the beneficiary groups targeted. At the time of the mission, field activities were being conducted by five people who had to cover all three project areas. Some service providers have not performed effectively. The FIL has not been set up. Income-generating activities and training to involve young people and women have not materialized. The strategies of research/action, development of product chains and economic interest groups and marketing for crop and livestock output were not properly elaborated. The agronomic and economic expertise (for individual and collective business plans) needed to promote production chains is lacking.

Recommendations: (i) Divide the component into two parts: one covering cross-cutting planning and the approach overall, and another covering agricultural development and natural resource management activities; (ii) have one or two responsible officers at the component level perform agro-economic activities; (iii) prepare business plans for key producers, a number of GIEs, groups and processing units; (iv) improve diagnostic techniques to better involve young people and women in needs identification and planning; (v) a range of income-generating activities and training aimed specifically at young people should be developed.

The ASF component was to have a service provider, but no contract was ever signed, reducing the component's effectiveness. Budget resources were not redistributed as a result. The lack of physical premises for ASFs has led to uncertainty in the safekeeping of funds at the village level.

Recommendations: i) build offices for ASFs; (ii) strengthen capacity in the ASF component of the UCP.

The rural infrastructure component encountered difficulties in connection with: (i) a population oriented towards schools and sanitary infrastructure rather than production infrastructure; (ii) delays in payments to entrepreneurs and service providers; (iii) large price increases; (iv) the departure and lack of replacement of component leaders; (v) the failure to complete the village tracks maintenance fund study, and the resulting lack of maintenance equipment at the village level; (vi) the lack of teachers in some schools.

Recommendations: (i) hire a component leader; (ii) provide training and equipment to village tracks maintenance committees; (iii) orient future investment towards profitable activities (development of valley bottoms, small livestock, silviculture, processing units, multi-use platforms, etc.).


1/ The evaluation was conducted by the Scanagri Denmark consulting firm based in Copenhagen, with the following  team members:  Mr Ole Olsen, Lead consultant, agronomist specializing in rural development; Ms Kristina Grosmann Due, sociologist; Mr Mamadou Baïlo Sidibe, rural infrastructure specialist; Mr Boubacar Barry, economist and rural credit specialist; Ms Djenabou Barry, sociologist. Mr F. Nichols, Lead Evaluator from the IFAD Office of Evaluation assisted in finalizing the work, conclusions and recommendations. During its work, the mission benefited from support provided by Mr Diallo, project coordinator, as well as his collaborators.

 

LANGUAGES: English, French

Upper East Region Land Conservation and Smallholder Rehabilitation Project - Phase II (2006)

Ghana  
October 2006

Interim Evaluation

Introduction

In accordance with the Evaluation Policy of the International Fund for Agricultural Development (IFAD), the Office of Evaluation (OE) conducted an Interim Evaluation1 of the Upper East Region Land Conservation and Smallholder Rehabilitation Project - Phase II in Ghana in May-June 2005, given the interest of both the Government of Ghana and IFAD's Western and Central Africa Division (PA) to proceed with further investments in the area. This evaluation adopts the standardized IFAD methodological framework for project evaluations.

Macro-economic and poverty indicators. Located in West Africa, Ghana has an estimated population of 20.5 million, of which 63% is rural. The structure of the economy is characterised by a large (in relative terms) services sector (42% of the total GDP), compared to 34% for agriculture and 24% for industry. It has an annual GDP per capita of USD 304 and the GDP growth has averaged 1.8% in the last ten years (i.e., below population growth) although this has increased recently. Agriculture continues to be the mainstay of the economy, employing about 60% of the labour force. Ghana is classified as 131st out of 175 countries according to the UNDP Human Development Index (2003). The percentage of households below USD 1 per day has been estimated at 44.8%, while the percentage of poor households at nearly 40% according to a national poverty line (World Bank, World Development Indicators, 2004).

Northern Ghana consists of three regions, Upper East (UER), Upper West (UWR) and Northern Region (NR). By many indicators, these regions are the poorest in Ghana and are indeed comparable in poverty to some of the poorest countries in the world. IFAD had projects in each of them: the Upper East Region Land Conservation and Smallholder Rehabilitation Project (LACOSREP) for the UER, the Upper West Agricultural Development Project (UWADEP) for UWR and the Northern Region Poverty Reduction Programme for the NR. The first phase of LACOSREP was designed in 1990, became effective by 1991/1992 and was given an interim evaluation in 1998. LACOSREP II was appraised in December 1998, became effective in January 2000 and will close in December 2006, after a one-year extension.

The most recent agricultural polices in Ghana are reflected in the: (i) Accelerated Agricultural Growth and Development Strategy (AAGDS); the Food and Agricultural Sector Development Policy (FASDEP); and the (ii) Ghana Poverty Reduction Strategy (GPRS) 2002-2004, currently under revision. The AAGDS broadly aims at the intensification and modernisation of agriculture, while the FASDEP further emphasizes the importance of food security. The GPRS of 2003 recognises that rural farmers and fishermen are particularly at risk, and specifically mentions Northern Ghana (UER, UWR and NR) as a locus of perennial food deficits. Women are identified as particularly disadvantaged in this context and instruments to promote gender equality are emphasised. Although the document notes the importance of environmental factors in increasing vulnerability, it does not propose concrete measures to reduce risk in agriculture. Beginning with 2003, some donors, in addition to their conventional "project approach", have also tested multi-donor budget support as an additional instrument in sectors such as health and education2. Agriculture and water sectors have not experienced major shifts to budget support.

The project area: serious problems of environmental degradation. LACOSREP II is located in the UER, the smallest region of Ghana (slightly over 3% of the total land area), with an estimated population of 920 000 people (80% employed in agriculture) and the highest population density in the country (104 persons/km2). Access to markets and off-farm opportunities is constrained in UER by poorly maintained feeder roads and lack of transportation services. According to the Ghana Living Standards Survey, the percentage of the population living in poverty is 88% in UER3. This is also reflected in the stunting rate for children below five years, which is higher than the national average (31.7% against 25%), although infant mortality rates are lower (33 per 1 000 against 68 per 1 000). UER is notable for its high levels of environmental degradation, deforestation and loss of soil cover, broadly as a result of extremely high population densities not accompanied by agricultural intensification.

The rationale for LACOSREP II was summarised in the Appraisal Report as: (i) strong existing demand for dam rehabilitation in rural communities; (ii) the potential for water user associations (WUA) to be sustained and assure food security in the region; and (iii) building on existing credit experience to establish effective mechanisms for rural financial institutions.

Basic project data. The total project cost was USD 13.9 million, out of which IFAD provided USD 11.5 m. As of mid-August 2005, 73.5% of the total loan amount had been disbursed. IFAD was the only international financier of the project, supervised by the United Nations Office for Project Services (UNOPS). The Ministry of Food and Agriculture (MoFA) was the agency responsible for the implementation at the national level, although the day-to-day management was entrusted to the regular regional staff of MoFA.

The objectives of LACOSREP II, as per appraisal, were to: (i) further develop irrigation in UER; (ii) increase productivity through farmer training and demonstrations of new technologies for increasing productivity of crops, livestock and fish; (iii) build the capacity of government institutions which provide technical and social services district and sub-district levels; (iv) construct rural infrastructure to reduce the female labour burden and take measures to mitigate the possible risks of health and negative environmental impacts. The components of LACOSREP II were as follows; (i) agricultural development (applied research, extension, livestock development, 15% of total base costs); (ii) water resources development (rehabilitation and construction of dams, environmental and human health protection, 33%); (iii) rural infrastructure (road improvement, dug wells and latrines, 19%) (iv) income-generating activities (rural credit, 18%); and (v) project organisation and management (14%).

Project interventions were to be implemented by: (i) MoFA district and regional staff; (ii) research specialists; (iii) Non-Governmental Organisations (NGOs) and agencies specialised in group formation; (iv) private sector; (v) consultants; and (vi) community-based organisations (CBOs). The Ghana Irrigation Development Authority (GIDA) was specified as the sole responsible agency for ensuring that the dams were built to high technical standards. The management of project interventions would be undertaken at district or regional level according to the scope and financing levels. The Project Coordination Unit (PCU) (integrated in the regional MoFA office) in particular would be responsible for all interventions that cut across districts, such as research.

Implementation status

Agricultural development: extension, farmer training and demonstrations. From 2000, the first year of the project, 197 of the 300 Farmer Training Demonstrations (FTDs) were conducted. Total participating farmers were 6 266, some 70% of the envisaged target, of which 3 898 were men and 2 546 women. However, it is important to note that the introduced technologies have spread well beyond the initial participating farmers and total adoption of FTD technologies is estimated as 8 756, clearly exceeding the original target of 6 000. Some progress has been made in marketing and processing: improved storage structures for onions has reduced loss from 50% plus to 25-40% over three to five months, when onions can be sold for as much as ¢300 000 compared to ¢80 000 per 73 kilo bag at harvest. Much remains to be done with more perishable species such as tomatoes.

Water resources development. This component experienced serious implementation delays. Out of the 32 dams to be constructed or rehabilitated, as of June 2005, contracts for 24 dams had been awarded. The rehabilitation of six dams and the construction of one dam had been completed, while the others were at various stages of construction. As a result, only 80 ha of the planned 372 ha are currently available for farmers (22% of planned irrigable area). Health and environmental interventions under LACOSREP II related principally to water-borne diseases, catchment protection, and the maintenance of soil fertility and agrobiodiversity. The introduction and development of composting has clearly been successful and the idea is apparently spreading even outside the project area. Environmental permits were issued by the Environmental Protection Agency to all LACOSREP II dam rehabilitations/constructions, although these were not renewed when delays in construction started. Catchment protection activities, including planting trees and creating bunds were carried out at LACOSREP I sites, and targets were largely achieved. Water quality analysis results carried out for hand-dug wells and in dams must be urgently made available.

Rural infrastructure. Spot improvements in roads (75 kilometres) of all designated areas were achieved during project life, although re-graveling was not carried out. Appraisal targets were generally met in the area of hand-dug wells and latrines, although many are not yet fitted with pumps (60%).

Income-generating activities. The component trained some 12 243 individuals in loan management (about 32% of the original target) and provided group loans without collateral, relying on group pressure to guarantee the repayment, to 10 251 individuals. No medium-term loan was granted for asset acquisition by any of the participating banks, apparently because they are uncomfortable with their management. The provision of financial services by the Agricultural Development Bank enabled savings to be mobilized and credit accessed from all six districts.

Performance of the project

Relevance

UER is the second poorest region of Ghana and overall living standards have hardly improved in the past ten years. The components of LACOSREP II are designed to add value to rural production and are, in general, valuable for the poor farmers. Clearly the funds of LACOSREP I and II allowed only partial geographical coverage. Ghana has signed up to various international undertakings to reduce poverty and LACOSREP II has this as its direct focus. The project is also in line with the goals for agriculture mentioned in policies and strategies such as AAGDS and FASDEP. Its value added consists of a clearer geographical targeting and a set of concrete measures to reduce poverty and help reduce or reverse environmental degradation. With the benefit of hindsight, two elements were not adequately focused. First, the project provided for a higher number of dams to be rehabilitated (23) than constructed ex novo (9). Communities without dams lack important production infrastructure (particularly in low rainfall areas) and need to be better targeted. Second, the experience from the sister project UWADEP in the adjacent region shows that it is possible to target specific irrigation interventions to special categories such as the blind and physically impaired. This experience deserves consideration for any further intervention4.

Effectiveness

In spite of the delays in the water resources (irrigation) component and the limited coverage of the income-generating activities (credit) component, LACOSREP has moved in the right direction in tackling the basic problems of the poor in UER. The development of the irrigation infrastructure and the improvement of agricultural practices through farmers' training and demonstration have benefited households in terms of improved income sources, assets and food security. The livestock development sub-component, with its emphasis on small ruminants and guinea fowl seems well placed, given the importance that they have in the project area. However, national policies dictating a switch to cost-recovery veterinary services create serious problems. Expenses of ¢ 1 000 to 2000 per livestock head are enough to discourage very poor farmers from vaccinating their livestock and diseases may wipe out years of investment.

Multi-component projects bear the risk of limited coordination between different interventions, which is also the case of LACOSREP II. For example, some farmers' training and demonstration activities can be very successful in themselves (such as introduction of improved varieties and certified seeds), but their effectiveness may be limited when inputs from other components are not provided timely (such as credit or seed at district and regional levels) or are not provided at all at the same sites. As different agencies are normally responsible for different components, the issue of synergies and integration between interventions needs to be carefully considered already at the design stage.

One of the objectives at appraisal was to construct infrastructure to alleviate women's work and provide mitigating measures for water-related diseases and negative environmental impact. Concerning the environmental impact, measures for catchment area protection and soil conservation proved successful in preventing erosion. Hand-dug wells were constructed to reduce women's workload in fetching water but they are not all functional and very little was done for protection from water-borne diseases.

Finally, concerning the objective of building capacity of formal and informal institutions to provide demand-driven social services, the picture is mixed. There are positive and important project achievements such as active and dynamic WUAs and Functional Literacy Groups (FLGs). On the other hand, there are two important limitations that need to be considered. First, there is a tendency for government field staff to spend a disproportionate amount of time in ‘sensitising' communities on benefits of certain interventions even when people are perfectly aware of their usefulness. One example of this comes from the irrigation component: some communities had filed application for dam construction or rehabilitation even before the start-up of LACOSREP II and yet it was decided to conduct a number of field workshops to explain how irrigation was important for them. Time and money would have been better invested in starting studies for infrastructure works and in involving representatives of the communities in those preparatory works. Second, the project contracted out some services to research institutions and non-governmental organisations (NGOs). However it did not really seize the opportunity to build a real ‘partnership' with them, it did not actively involve them in assessing community needs and testing new technology that was directly responsive to the elicited needs. Some NGOs had accumulated experience in this sense that could have been built upon.

Efficiency

In this report the notion of efficiency is explored mainly for the irrigation and the rural finance components. In the case of irrigation, unit costs of construction (per ha) are compared with benchmarks in the country. In the case of the rural finance component, the administrative cost of providing a dollar of credit is compared with a peer group of microfinance institutions in the region. The unit cost of dam construction for LACOSREP II dams varied from USD 477 to 1338.7/ha5. A study prepared by the Food and Agriculture Organisation of the United Nations (1998) showed ranges of costs per ha between USD 400 and 5 000 per ha in Ghana. This suggests that LACOSREP II construction costs were relatively inexpensive compared to typical benchmarks. In terms of water conservation, new technology based on piped systems has been introduced in northern Ghana by international NGOs and this technology should be considered carefully in future interventions as it would improve efficiency in water conveyance and distribution.

As far as rural finance is concerned, administrative costs of lending for the participating banks are about one-third of the average of a group of rural finance institutions in the region (Microbanking Bulletin 2004). From a cost-effectiveness point of view, the project appears to perform well, although water delivery efficiency can be improved, quality of water infrastructure needs to be monitored because many dams have not been completed, and the rural finance component exhibits problems in credit discipline (low repayment rates, as explained further below).

Rural poverty impact

Methods. In assessing the project impact, the evaluation triangulated among multiple sources: (i) the available M&E data, (ii) a quantitative survey of beneficiaries and non-beneficiaries; (iii) a qualitative survey of five dam sites; (iv) an ad hoc survey of non-beneficiaries; and (v) participants' observations as recorded by mission members. Following the evaluation policy, the project conducted a self-assessment exercise which was also considered by the evaluation team. Detailed results are presented in the main report.

Significant impact on physical and financial assets. The evaluation surveys compare households that have been assisted by LACOSREP with households that have not benefited of such assistance (control group). Data suggests that assets of project users (both household and agricultural assets) have increased significantly in the past years, while increases for the control group were less evident. Qualitative interviews during the field visits confirm that this is the case. Those households that have received financial services from participating banks have reported benefits in terms of better opportunities for investments in trading, farming but also activities that do not directly generate income but contribute to household welfare, such as health and schooling (as further explained below).

Important impact on farming practices and literacy, limited progress on human health issues. The evaluation surveys provided quantitative evidence of increased adoption of improved farming practices, including composting and improved seeds. The predominant approach in relation to farmers' knowledge was that of the field school which focused on practical cases and problems. Improved small ruminant management, including housing, nutrition and health care has been taken up, by farmers and communities involved in the livestock components. The promotion of FLGs was an element not foreseen in the original design, which was strongly appreciated by the communities and also helped strengthen social capital (as explained below). Instead, limited progress was made concerning human health (risk of water borne diseases, such as schistosomiasis and malaria), which had been flagged as a significant issue towards the end of LACOSREP I. In the course of visits to WUAs, the team evaluation found no evidence of training or advice to the communities.

Social capital and empowerment. Training programmes and activities were carried out during LACOSREP II for the newly established WUAs of LACOSREP II as well as for groups created under LACOSREP I. There is every sign that many WUAs are vibrant organisations capable of handling their affairs, although some require further training in irrigation practices, such as irrigation scheduling and methods of field water application. The introduction of FLGs enhanced the dynamism of groups. One weakness of WUAs is the limited amount of financial resources they can raise, compared to major maintenance needs. This is not a surprising result but should be taken into consideration by MoFA and the donors when budgeting for project maintenance costs.

Enhanced food security. In the absence of a baseline anthropometric survey, the evaluation has to rely on qualitative perceptions elicited during interviews, as well on indirect evidence, such as the increased yields due to improved varieties or cropping rates in the dry season. All the beneficiaries interviewed acknowledged that the project has enhanced their food security, both through access to credit (consumption smoothing effect), cash earned during the dry season and better skills in marketing. Some use their profits or part of the loan to stock food for the hunger period. While some report having adequate food during the hunger period, others have the food scarcity of the hunger period reduced.

Environmental impact. The serious environmental degradation problems in UER required LACOSREP II to emphasise soil and water conservation. The project has had a considerable positive impact both on direct increase of soil productivity in project areas and improving participant and neighbouring communities' awareness about soil conservation. Catchment protection activities including tree planting and creating bunds were carried out at LACOSREP I sites, and the targets were largely achieved.

Impact on institutions and policies is generally limited. Village institutions such as WUAs and FLGs are clearly viable but it is more difficult to see a major shift in the attitudes of government staff. For example, the time taken at the inception of LACOSREP II ‘sensitising' communities to the need for dams, when many had lodged applications with the Regional Assembly as early as 1998 suggests that much needs to be done in this area. NGOs were used principally to carry out services for the LACOSREP II, such as the use of Rural Aid in hand-dug wells and ActionAid in the REFLECT literacy strategy. They were not generally involved in testing new technologies or assessing community well-being, in spite of the experience that they had accumulated. Change in mentality takes time, but IFAD could help by facilitating communication and dissemination of experiences from the ground. The project did not promote the strengthening of rural finance institutions: banks were seen more as conduits for credit rather than institutions that need to become financially sustainable. Finally, an area yet to be addressed relates to policy dialogue on cost-recovery requirements for livestock vaccination. As previously noted, such requirements exclude many farmers from access to animal health services.

Gender. LACOSREP II employed a gender officer on a contract basis to ensure that the objectives of appraisal were met, and this has been an effective strategy. Women were not traditionally land owners in this region, but the WUA system has given them direct access to irrigated land. The FLs have also provided an arena for women to co-operate and organise collective income generation.

Sustainability threats can be considered under three key dimensions: (i) maintenance of dams; (ii) environmental sustainability; and (iii) financial sustainability of rural finance operations. Responsibility for maintenance of dams had not been clearly spelled out: some maintenance operations are beyond the operational and financial capacity of WUAs and sometimes even local government budget availability. In terms of environmental sustainability, there are technical innovations in agriculture that can be continued even after project support, such as composting and vegetable production. Irrigation increases cropping rates and the extraction of soil nutrients. This could be countered by adequate agroforestry packages and the integration of leguminous crops into rainfed cropping cycles, and there is a need for more emphasis on these. Concerning the rural finance component, financial sustainability risks stem from two reasons: first, the repayment rates are low because of high transaction costs (bank offices are far away from their clients, untrained bank staff and insufficient products to serve the poor); and secondly, the banks are forced to lend at interest rates that do not allow them to cover their costs.

Innovations, scaling-up and replicability. The most successful innovations promoted by the project are those that can spread from farmer to farmer with little capital investment. Guinea fowl raising, composting, the use of neem for crop protection and formation of literacy groups were not diffused in the UER before LACOSREP II but are now being increasingly adopted. They are all examples of such innovations, for which there is evidence of this type of spread, since they have been replicated even outside the project area. Several interventions (improved livestock care, effective response to plant pathogens) are within the technical and financial capacity of MoFA and can be replicated without additional donor funding. However, to construct new dams or rehabilitate the existing ones, donor funds are needed.

The overall impact of LACOSREP II on beneficiary communities has been considerable in the areas of food security, income generation, corporateness, literacy and promotion of gender issues, in spite of implementation constraints and delays. The achievements of both phases of LACOSREP should be seen against the trend of increasing poverty and environmental degradation in the UER.

Performance of partners

IFAD has rightly insisted on the LACOSREP approach which has important potentials for poverty reduction in the UER. There are, however, certain flaws in the original project design which deserve further reflection: first the whole rural finance component did little to enhance the financial sustainability of the participating banks; second, the limited degree of integration between components (which was partly dealt with during the implementation phase); and third, the over-reliance on the GIDA to screen private contractors for the irrigation component. This created delays in the execution and did not ensure high quality of works. The lack of IFAD field presence in Ghana constrains implementation support as well as policy dialogue and coordination with other donors. Problems such as the inadequate performance of GIDA in the irrigation component are difficult to address through a two-week UNOPS mission per year. On the other hand, experiences such as that of LACOSREP I and II should be more systematically brought to the attention of other donors as well as local research institutions and NGOs active in the country.

UNOPS has been responsible for the supervision of LACOSREP, with yearly visits of about ten days. Reportedly, these missions have been conscientiously carried out and the detailed reports were critical but fair. However, there was a perception by MoFA staff that in recent years it was more difficult to have a dialogue with the UNOPS team, that the missions were more adversarial, perhaps due to a sense of frustration with delayed implementation.

The Government. In general, the PCU/MoFA performed creditably, and problems with day-to-day running have been resolved successfully in many cases. All types of training and sensitisation were carried out in a timely manner and there was a certain adaptation and flexibility in their implementation. In the area of irrigation infrastructure, the problems of managing contractors and an over-reliance on GIDA should have been resolved much earlier. In terms of monitoring and evaluation (M&E) activities, the monitoring part was quite accurate, while the evaluation of the socio-economic impact weak. However, the contribution of GIDA as the sole consultant for the irrigation infrastructure on the project was a weak one, causing many problems in project execution, as well as delays in implementation. National policies on cost-recovery of livestock vaccination do not represent a conducive environment for projects such as LACOSREP II. They are among the root causes of low vaccination coverage and expose farmers to the risk of livestock losses to epidemics.

Research and extension. The project collaborated with a number of research organisations on small ruminant husbandry, guinea fowl husbandry, improved crop and pest management in rice and improved method of parboiling rice. The performance of the four research institutes involved, Animal Research Institute, Tamale, University of Ghana; Crop Research Institute, Kumasi; and Food Institute, Accra, were all considered satisfactory to good by the project. In most cases however, the tendency was for the project to contract out individual survey assignments rather than establishing a partnership, which involves needs assessment, follow up and modification/direction for future applied research. The performance of the major research partner as envisaged at project appraisal, the Savannah Agricultural Research Institute (SARI), was weak. Despite the considerable infrastructure support given to the institute during the previous phase, it was not able to give effective advice to counter the serious problem of the tomato disease complex.

Banks. The role of the Central Bank of Ghana was to set up and manage the disbursements and replenishment of the Revolving Credit Fund for the participating banks. Although it was required that the clearing accounts of the participating banks be debited on due dates, as at March 2005, all the participating banks had balances overdue since December 2004. As explained above, the performance of the participating banks was weak, due to low repayment rates. This resulted in poor portfolio quality. The lowest rate of 27% for Portfolio at Risk (>30 days) achieved by Naara Rural Bank is weak by any standard. Building sustainable financial institutions was not a major ingredient in the project design.

Private contractors. Timeliness and quality of work by local contractors was a major problem throughout LACOSREP II. Procurement procedures for contractual services were adopted to conform with the World Bank standards and to ensure transparency and objective choice, but with weak results.

NGOs. Rural Aid, the main NGO concerned with excavation of hand-dug wells and household latrines, generally performed satisfactorily. ActionAid, whose methodology was used in the formation of FLGs was evidently successful. Instead, the three NGOs which worked in partnership with the rural banks had limited experience in rural finance and failed to provide continued assistance.

Overall assessment

The most important conclusion is that overall, and in spite of implementation delays in the second phase, both phases of LACOSREP have benefited poor communities in UER, have increased levels of food security, enhanced capacity for collective action, improved material well-being and effectively heightened gender awareness. However, dilatory construction and completion of dams and the likelihood that many will be unfinished after closure of the project is a serious problem. Central government officers in Accra have argued that development investments in Northern Ghana have been ineffective. However, this evaluation is of the opinion that the relative success of LACOSREP II rather shows that there are experiences which deserve further support, clearly within an improved framework.

Insights and recommendations

Insights

Integration and sequencing of components. If IFAD is to consider further investment in UER, then the evaluation of LACOSREP II suggests the rethinking of some elements in project design. A key problem is that projects with many components, but without a strong integrative strategy, are open to activities being carried out with no linkages, which result in high management costs. Project design should consider sequencing more carefully.

Considerations of equity. LACOSREP II and comparable projects such as UWADEP also raise broader concerns. Focusing on the rehabilitation of existing infrastructure does little for communities that have not benefited from outside assistance in the past. Similarly, protection of special categories such as the blind or the physically impaired was not part of LACOSREP II design, but experience from UWR shows that this can be made to work and it should be included in future project design.

What type of intervention? In Ghana, in some sectors such as health and education, donors have started to fund multi-donor budget support initiatives for national strategies. Although the pressure is not yet strong in the agriculture and water sector, the question is whether area development intervention deserves receiving further funding in the future. The experience of LACOSREP illustrates that there is a very significant need on the ground for projects and programmes that clearly focus on a well-defined geographical area and deliver services and inputs to impoverished farmers. While the area development focus can be retained, as previously noted, IFAD should promote a stronger integrative strategy, not only through a more careful consultation at the design phase but also by promoting better discussion and dissemination of ‘lessons from the ground'. Moreover, rural finance interventions will not be effective unless they aim at strengthening rural finance institutions. One option could even consist of having a dedicated programme for rural finance. If this choice is made, clearly the dovetailing and synchronising of rural finance with other interventions becomes a crucial issue.

Whatever the problems, the contention that development interventions in UER have not resulted in improved wellbeing is not supported by the empirical data gathered by the evaluation. Moreover, equity considerations, the contribution of labour from UER to national development and the Government's ratification of various poverty agendas all argue for continuing and indeed expanded investment, within an improved framework. In future investments, the priority of improving the design should be matched by the search of strategic partners, as the availability of IFAD resources allows only limited coverage.

Recommendations

Immediate tasks: completion of irrigation infrastructure and health issues. LACOSREP II will close in about a year and a significant part of the dam infrastructure remains unfinished. Completion of existing works is a major priority, but will only be a useful exercise if closer supervision of works is introduced, preferably using alternative arrangements (as elaborated upon). Health issues (especially the risk of water-borne diseases) need to be addressed to avoid that health conditions of the beneficiaries deteriorate after project closure.

Communicating and discussing project experience as a contribution to policy dialogue. The two phases of LACOSREP offer important lessons that are grounded in the reality of the field. These experiences should be documented and widely discussed not only at the district and regional levels, but also at the national level, taking the opportunity of the donors' coordination group in the agricultural sector. This report has highlighted areas of weak institutional impact, resulting, for example, in little evidence of interactive approaches where ideas and concepts from the village make their way to project design. The same can be said in the case of livestock vaccination: donor-driven policies of cost-recovery were clearly not working because of widespread cash shortages with consequent economic losses due to animal deaths. On the other hand, a number of good practices (e.g., a strong commitment by WUAs, soil and water management) have emerged from the two phases of LACOSREP and more can be learned from other experiences. A first step in policy dialogue would be to facilitate discussion of the lessons stemming from the project experience at the local (district, region) and national levels.

M&E and supervision. Collection of quantitative data on project delivery was relatively accurate. The missing part was the assessment of impact on household welfare, food security and health. Supervision was conscientiously carried out within the constraints of the arrangements between UNOPS and IFAD which allow for two-week mission per year only. Both these findings relate to the "discrete" and short-term nature of implementation support arrangements within IFAD project. For effective M&E and project implementation, project staff would rather develop a more responsive relationship with adviser(s)/facilitator(s) who would be available on a more regular basis.

Irrigation

Quality control and phased contracting of consultancy services. Opening the consulting for the irrigation infrastructure to a wide range of professional companies should be non-negotiable. Proposals should be sought from other qualified consultants to compete for the assignment in the downstream phase even where the consultant has performed satisfactorily. Consulting services should be divided into phases according to the project implementation schedule, and contracts signed separately, subject to satisfactory performance. In order to better understand bottlenecks, IFAD may consider conducting an audit of contract awarding during project implementation.

Dam construction technology. New methods of dams construction are available that save a considerable amount of water. ‘Closed' systems are being introduced by international NGOs and this technology should be considered carefully in the future interventions.

Pumping water from the White Volta River for riverside horticulture was introduced in the final year of the project. Although not in the project design, it has been introduced to allow cultivation of three crops per year. This method is simple and effective, but requires pumps and diesel fuel as inputs. However, the nutritional and income-generating benefits suggest that small-scale credit could rapidly extend the benefits to a wide range of farmers, as has been shown in other countries such as Nigeria.

Agricultural support

Action research and extension. The present evaluation has highlighted the need for stronger partnerships between projects, research organisations and NGOs that would translate into support for the most needed technological packages. MoFA and the project unit could also facilitate farmer-to-farmer communication through cross visits and a more extensive use of the already existing radio programming.

Processing and marketing. Crop diversification, dissemination of new techniques in marketing and a variety of crop processing strategies could rapidly increase incomes and reduce nutritional insecurity in UER. A review of these issues and action should be taken by MoFA because much knowledge already exists in neighbouring countries, notably Burkina Faso, so some type of farmer exchange is recommended.

Additional Area for Streamlining: FLGs

FLGs, originally not included in the project design, have seen considerable success, both increasing numeracy and literacy and establishing solidarity among groups for other purposes such as collective work and micro-finance. Strategies to further develop FLGs should be worked out with NGOs, such as ActionAid, that have piloted them.

Rural Finance Issues

While credit was appreciated by the beneficiaries, coverage was limited and the project did not significantly contribute to promoting sustainable rural finance institutions. It is important that: (i) rural banks be allowed to apply interest rates that cover all costs and allow for profits; (ii) partners, including the Bank of Ghana, contribute to better coordination and regulation; (iii) training be provided to participating banks' staff using regional rural finance hubs; (iv) discussions be held with participating banks on available techniques and products that can help reduce transaction costs in rural areas; and (v) the rural finance component is fine-tuned and well sequenced with other components.


1/ The Office of Evaluation of IFAD conducted an evaluation mission of LACOSREP II and UWADEP from 23rd May to 30th June 2005.  Field visits in the UER took place between 26th May and 10th June. Sites of both LACOSREP I and II were visited, the former to better understand key sustainability issues. The mission members were Mr Roger Blench (Team Leader), Mr David Andah (Credit and Micro-finance), Ms Liz Kiff (Agricultural Extension) and Ms Gordana Kranjac (Water resources and Rural Infrastructure). Preliminary quantitative and qualitative surveys were carried out by Mr Hippolite Bayor and Mr Edward Aboagye in early 2005. An ad hoc survey of non participant households was conducted in concomitance with the mission, under the direction of the Team Leader. Mr Fabrizio Felloni (Lead Evaluator, IFAD-OE) designed the evaluation methodology, made a pre-evaluation visit in April 2005, accompanied the mission for its first and final days in Ghana and supervised the evaluation process throughout.  An aide-mémoire and an associated PowerPoint presentation were circulated at a workshop in Bolgatanga on 10 June 2005 under the chairmanship of the Regional Minister.  A final presentation of the first findings from LACOSREP II and UWADEP was made in Accra on June 30th. The mission is grateful to national and regional authorities as well as to the project staff for their support.

2/ The ten donors involved in multi-donor budget support in Ghana are: the African Development Bank, Canada, Denmark, the European Union, France, Germany, the Netherlands, Switzerland, the World Bank and the United Kingdom.  Japan, the United States and the United Nations are observers.

3/ Consumption-based estimates, GPRS, 2003. Usual caveats on monetary-based estimates for rural areas apply also to this case.

4/ UWADEP, the sister-project in UWR, has at least one dam site, Karni, where assistance to the blind, disabled and single mothers is a major element, showing that this can be made to work.

5/ There are no precise data for maintenance costs.




 

 

LANGUAGES: English

Economic Development of Poor Rural Communities Project (2006)

Venezuela (Bolivarian Republic of)  
September 2006

Interim Evaluation1

Background. PRODECOP was approved by the Executive Board of the International Fund for Agricultural Development (IFAD) in September 1996 (Loan 427-VE). The loan entered into effect in June 1998 with an estimated implementation period of six years. The total cost of PRODECOP was estimated at USD 24.4 million, of which IFAD was to finance USD 12.0 million, the Government of Venezuela (GoV) USD 8.7 million, the co-financing institution 2.8 million and the beneficiaries USD 744,000. The cooperating institution is the Andean Development Corporation (CAF). The implementing agency is the Foundation for Training and Applied Research in Agrarian Reform (CIARA), with oversight from the Rural Development Institute (INDER), which is part of the Ministry of Popular Economy (MINEP).

The interim evaluation mission visited the country between 16 May and 7 June 2005. Its terms of reference were to evaluate the project's operations, draw lessons learned applicable to the design of future projects, make recommendations for implementation during the remaining life of the IFAD loan and analyse the advisability of formulating and carrying out a second phase of the project under evaluation. The mission held working meetings in Caracas with authorities from the MINEP and from CIARA, the Ministry of Planning and Development and CAF. The mission made an extensive tour of the project area, covering seven Venezuelan states and visiting 20 municipalities. Interviews were conducted with State agricultural officials and with municipal authorities within the project area, as well as conversations with beneficiaries.

Brief project description: Concept, objectives, goals and components. The diagnostic assessment and project concept were based on the assumption that the major constraints to improving living conditions and production among the target group lay in the lack of community organization and the lack of access to sustainable rural financial services. Accordingly, the following measures were proposed: (i) a large-scale training effort focusing on promoting and establishing social and economic/productive organizations at the community level; (ii) close collaboration with the Agricultural Extension Programme (PREA) being implemented by CIARA with World Bank financing in the same areas as PRODECOP, to ensure technical assistance for agriculture; (iii) a Community Initiatives Fund (FIC) to finance community projects proposed by organizations; and (iv) an ambitious programme to set up rural savings and loan cooperatives (i.e. rural credit unions) to mobilize savings among the target group and empower beneficiaries in credit management, together with a complex rediscount and guarantee system to provide these financial units with access to commercial lines of credit. Organizing beneficiaries was assigned priority as the most significant activity, not only to empower other actions but also to achieve the objective of "ciudadanizar" 2 rural populations to involve them in policy-setting on decentralization. Moreover, particular emphasis was placed on gender in all project activities, to ensure that women would participate directly. PRODECOP was designed to cover a vast geographical area that takes in eight states and 39 municipalities.

The overall project objective is "to improve the quality of life and economy for poor farming families and small-scale producers by better positioning them within local development processes, increasing their farming income and improving prevailing socioeconomic conditions". The project's specific objectives are to: (a) provide training and technical assistance for rural poor communities; (b) create and strengthen participation mechanisms among rural poor communities and municipal and state governments; (c) create and strengthen social, economic and financial local organizations; (d) improve family incomes; (e) support policies for decentralization and strengthening of municipal governments. The project's ultimate goal is to raise community self-awareness. In order to achieve these objectives, three components were planned: (1) Training for development and citizen participation: to build technical and social capacity among project beneficiaries and the technical team, as well as municipal and state officials, to enable them to play an active role in social and economic development; (2) Financial services and rural financial system with three main objectives: (i) to provide financial services in support of production; (ii) to organize sustainable community-based rural financial institutions to provide such services; (iii) to initiate and consolidate linkages between such rural financial institutions and the banking system and other financial institutions, through the establishment and operation of guarantee and rediscount funds; and (3) Monitoring and evaluation: the monitoring, evaluation and management unit to oversee implementation of the components was to be set up within the Project Executing Unit (PEU) at CIARA.

Project targets included 760 communities served, 334 civil associations organized and 476 multiple interest groups for the first component. For the second component, targets included 198 groups planning credit unions and 150 credit unions. Plans called for 19 003 rural families to benefit under the training component for citizen development and participation, 15 000 families under joint actions in technical assistance for production by the project with PREA, and 9 000 members of community savings and loan organizations (150 "mature" entities) under the financial services and rural financial system component. Among these, plans called for 8 000 women to benefit and for 250 municipal employees to be trained. The project's gender approach was defined on the basis of an affirmative action strategy. Accordingly, the gender focus cuts across all project actions. No estimates were included in the project design with respect to project returns or expected increases in beneficiary income, nor was a logical framework prepared.

Results obtained. The project has met the original targets for reaching users under the major components. The training component benefited at least 13 000 families or 15 542 persons (82% of the target), and the financial services component exceeded the target with 11 458 users (121% of the target of 9 500 users). The number of communities reached is slightly higher than planned (779 versus 760), and the number of organizations registered is significantly higher than the target (655 civil society organizations compared to the 334 projected). Credit unions established number 151 (compared to the target of 150). The mission concluded that, given conditions at start-up, the project design overestimated the possibility of consolidating these financial entities during the implementation period. According to available information, some 9 000 women have used the project services, exceeding the target. The project has made an important contribution to setting up, training and developing private providers of technical assistance in all areas concerned.

The targets set for beneficiaries of technical assistance programmes were not met, nor were those for training municipal officials. Project performance was virtually nil on targets for loans to credit unions and establishment of a guarantee fund. The mission reiterates that the project design overestimated the viability of reaching these goals, given conditions prevailing at start-up. Finally, the project targets and resources to be financed under the FIC were not met, although disbursements were made and many projects were funded.

Financial performance. From the point of view of financial performance, project disbursements by end-2004 stood at 56% of the total. By source of financing, the Government of Venezuela had disbursed 75% of its contribution, the CAF 100% of its loan and IFAD only 34.7% of its loan (with 40% projected for mid-2005). Virtually all training resources were disbursed; outlays for project management exceeded projections by 20%; and 11% more than planned was disbursed for monitoring and evaluation. The FIC disbursed only 32% of projected resources. Under the financial services component, only 20% of planned disbursements were made since the guarantee and rediscount funds did not materialize. Given that the proceeds of the IFAD loan were to go to these funds, this explains the low disbursement rate for the loan. On the other hand, beneficiary contributions were greater than expected.

Project performance and impact on poverty and sustainability. Relevance of objectives. The overall objective of the project was not only highly relevant at the time of design but continues to be relevant, perhaps even more so, at the present time. The far-reaching changes in the country's economy in the past three years have laid a better foundation for increased participation by rural poor communities in public spending decisions at the national and local levels. Similarly, the changes seen in the political and institutional setting in the past five years are much more favourable towards an increase in grassroots local participation and achieving substantial improvements in socioeconomic conditions for the poorest social strata. Effectiveness. A review of PRODECOP's impact shows that (cf. the PRODECOP effectiveness matrix in appendix 5) the project was highly to very highly effective in terms of impact on: (i) social capital and empowering the population for action; (ii) human capital relating to better information and training and greater access to primary education; (iii) physical community assets; (iv) financial assets and greater access to financial services; and (v) institutions (particularly financial), policies and regulatory frameworks. On the other hand, PRODECOP was not very effective in connection with: (i) productive physical assets of families; (ii) producer relations with markets; (iii) agricultural output and food security; and (iv) the environment and the community resource base. Finally, PRODECOP was moderately effective in: (i) increasing physical consumer assets of families; (ii) human capital relating to health conditions, access to drinking water and the workload of women and children; and (iii) reducing environmental risk. Efficiency. Based on an analysis of the broad range of coverage achieved by PRODECOP, and an acknowledgement that the original design objectives were incorrectly formulated, the project's overall efficiency can unquestionably be deemed satisfactory.

PRODECOP is a good illustration of the virtually unequivocal relationship between design and proper implementation of components and activities, as well as achieving the expected positive impact. The results achieved under the training component for citizen development and participation generated substantial impact on the ability of organized communities to manage and obtain much greater investments and services in basic and social infrastructure. This social capital formation in turn generated a very positive impact on physical community assets, human capital relating to education and training, and to an appreciable degree health, living conditions and work by women and children. Also, the excellent implementation of promotion, training, establishment, operation and support activities for rural credit unions generated a notable impact on the installed network of such institutions and on their sustainable and increasingly autonomous operations. This in turn gave rise to a marked impact on the creation of local financial institutions and on access to credit, with the resulting impact on increasing physical consumer assets in households, on food security and on the sustainability of many agricultural activities.

The PRODECOP experience also shows that deficiencies in the design and/or implementation of some components can lead to little or no positive impact being achieved. An inadequate design of agricultural technical assistance facilities (contingent, without sufficient assurances, upon assumed coordination with another project financed by another international agency), aggravated by deficiencies in implementation, led to no impact being achieved on agricultural productivity, technological innovation, and finally, family farming income. Moreover, the financial services component design regrettably allocated most of the IFAD loan proceeds to creating the proposed guarantee and rediscount funds. Although these were consistent in principle with setting up a rural financial system with connections to private commercial banking, they were clearly premature given the level of development of local financial institutions, and not viable for the existing institutional framework. These deficiencies were reflected in the fact that the component had virtually no impact on the magnitude of financing for technology investments and innovations in the agriculture sector, no impact on the physical productive assets of the beneficiary population, and consequently minimal impact on farming income for project beneficiaries.

Impact on poverty. PRODECOP has generated a series of effects on the rural communities directly involved in its implementation, as well as on other economic and social agents linked to these communities and/or the relevant institutions. The mission concludes that no adverse impact was detected on conditions of poverty for the population concerned. On the contrary, the project led to a generally positive impact on various aspects of living and production conditions for the users of its services.

Sustainability. PRODECOP has generated a number of changes and effects with an undeniably positive impact on the economic and social wellbeing of the population. The permanence of this positive impact is contingent upon a series of constraints and conditions for sustainability. The sustainability of impact on social capital and the population's empowerment to interact with the authorities is clear since these are virtually irreversible attributes. The rural credit unions are undoubtedly the most sustainable structures generated by PRODECOP. This is determined by the management and operating model governing these local financial institutions to the extent that they maintain their dimensions and modus operandi.

Overall project evaluation and findings. PRODECOP may be summed up as a project with satisfactory implementation. This should be understood as a positive judgment because the project obtained results and generated a significant beneficial impact for the rural poor despite deficiencies in design and a number of contextual obstacles. Moreover, PRODECOP has generated a sound and sustainable basis for maintaining and expanding a positive dynamic for rural development, a high level of community participation and an expansion in local financial institutions. The project should be considered to be a successful first step, yet to be concluded, in the necessary process to fully achieve the original objectives. The main conclusions are as follows:3

PRODECOP has achieved the most important major objective in qualitative terms: the "ciudadanización" of tens of thousands of rural poor. Through training and promotion activities by community organizations, the users have "recognized one another", "grouped together", "awakened" and learned to exercise their citizens' right to petition the authorities in an organized and articulate manner. This formation of "social capital" is the project's most significant accomplishment.

PRODECOP was designed with relevant objectives based on a diagnosis that correctly placed emphasis on the scarcity of social capital in rural areas and, in particular, the lack of rural financial services, as the main obstacles to comprehensive development of the country's rural poor communities. Paradoxically, the design of instruments to address these issues included both extremely well-founded proposals and others that were ill-advised and not viable. The latter have had an adverse impact on the conditions of production for beneficiaries and above all on the disbursement of IFAD loan proceeds. Most of the unused funds had been allocated in design to set up a complex system of rural financial services (including a guarantee fund and a rediscount fund) that is not considered to be viable given the country's historical, geographical and institutional conditions. Nor has PRODECOP taken actions having any substantial impact on the conditions of production for beneficiaries. The failure of working agreements with PREA to materialize —having been inadequately provided for in the design and without explicit coordination arrangements with the World Bank— led to virtually no activities being carried out in this respect.

PRODECOP has been implemented with great dynamism and increasing effectiveness and efficiency, covering vast geographical areas with heterogeneous and complex characteristics that have not always been sufficiently taken into account. Actions in the indigenous areas included and a large part of the Eastern Area ought to have been strengthened further to ensure achievements similar to those of other regions.

PRODECOP achieved significant coverage, with between 16 000 and 20 000 families having used its services, mainly training. This is a remarkable achievement given the starting point of virtually no services in support of the country's rural poor population.

PRODECOP has mobilized investments and services in basic and social infrastructure in an amount and of a quality well above original plans and estimates. A large number of projects in basic and social infrastructure and services have either been carried out or are in progress in many rural communities served by the project. These investments, financed with public funding at various levels, would not have reached the communities without PRODECOP's training and organization activities.

The PRODECOP rural credit unions are the most successful experience in rural microfinance in the country and probably in the Latin American region. Rescaling the objectives set in the project design for the financial services component on a more realistic basis during PRODECOP implementation enabled dozens of rural credit unions to be promoted and set up effectively and sustainably. Their greatest significance lies not so much in the amount of resources tapped (approximately USD 800 000 in 151 rural credit unions) or mobilized (lending for approximately USD 3 million), but rather that these resources actually represent the savings of the rural poor and are being managed and administered by them. The dynamics of growth and expansion among many of the rural credit unions has laid the groundwork for establishing more complex networks and incipient financial markets in the rural environment. Rural credit unions have transcended the financial sphere to become grass-roots social organizations that are much sounder and more sustainable than any other. The design adopted for these mechanisms is favouring the inclusion of the poorest in society, as well as gender equity.

PRODECOP showed that mobilizing savings and loans from the rural poor is a complex process that requires a much longer maturation period than assumed in project design. It also showed that rural credit unions face constraints on their growth and soundness because they tap savings on a scale and timing that may not be compatible with the investment financing that can bring about a structural transformation in current technologies. This experience indicates that the use of such mechanisms is an effective and efficient response to the need for savings in terms of transactions and safeguards, as well as the need for credit to meet short-term demand. This provides a background for criticism of the design of overly ambitious proposals that link these local institutions to commercial banks prematurely. PRODECOP generates a series of questions about the future of such instruments and the need to come up with mechanisms to complement investment financing.

PRODECOP adopted several innovations during implementation that improved upon the design and enabled the project to adapt to a changing context and realities not properly perceived during the formulation stage. However, neither PRODECOP nor IFAD took sufficient action during implementation to reformulate crucial aspects of design that had a significant impact on loan disbursements and an impact on agricultural production by the beneficiaries. A series of innovative training methods in citizen participation characterized implementation of the component from the outset. Subsequently, several approaches to activities in support of communities and organizations enabled the project to have a significant impact. Similarly, setting up and supporting rural credit unions was done using innovative methods and systems not provided for in the original design. However, the design defects mentioned under the financial services component ought to have been formally recognized much earlier by the GoV and communicated to the cooperating institution and IFAD. Moreover, the IFAD mid-term review mission (2002) ought to have recognized the design failings rather than continuing to insist on the original design. This would have made it possible to take steps to reallocate these funds to other project activities. Also, the limited use of the FIC, explained by years of budget constraints at the national level, reveals limitations in negotiations to obtain budget authority to carry out the project properly. Similarly, the failure of the proposed agreement with PREA was identified early on in project implementation. Nevertheless, appropriate steps were not taken to strengthen these aspects (e.g., by including agricultural extension workers in teams of field promoters in select areas). Only at the end of implementation, in 2003 and 2004, were some efforts made to address this objective, and they remained insufficient.

PRODECOP's actions in indigenous areas and with indigenous groups present a number of failings as well as some successes. The project design called for special treatment for indigenous areas that would take into account the cultural characteristics of Warao and Kariña populations. Although a differentiated approach was developed for each ethnic group, the treatment for indigenous areas was similar to that of the project areas as a whole. The mission observed that, with the exception of a CAF supervision report, very few people have visited the area either from the missions or the PEU. Constant turnover in promoters occurred throughout the life of the project. Nevertheless, the mission observed the enormous value placed by these populations on the project actions. This led to an appreciation of the importance of strengthening socio-community organizations and implementing social projects in this area.

The strengthening of mutual relations between PRODECOP and town halls and local governments remains a pending issue. Although contextual variables prevented these objectives from being fully achieved, this mandate should nevertheless be taken up again in future stages. The sustainability of citizen participation at the local level is fundamentally contingent upon these dimensions of local development.

The gender approach was adopted and implemented successfully by PRODECOP. PRODECOP developed strategies for innovative and creative action that fostered gender equity naturally. In this way, the successes of the rural credit unions' organizational design generated a significant feminine presence and leadership. Moreover, (a) affirmative action in hiring PEU staff empowered the project in terms of equity and inclusion; (b) including women in project decision-making led to improvements in the quality of life for the beneficiary population as a whole; (c) disaggregating information by sex was a way of allowing the project to add value to its intervention under a differentiated vision of effects and impact; (d) promoting active participation by women in financial organizations lent value to their productive work; (e) purchasing shares and accessing credit unions in their own names placed women in an equal position to men for decision-making on the organizations' administration and management; (f) women's participation in diagnostic assessments and community planning provided a more comprehensive view of problems and their possible solutions.

Performance by the PEU has been more than satisfactory. The project enjoyed continuity in management and among most professional and technical staff for close to six years. This continuity enabled activity lines, consistent approaches and continuous actions to be maintained to benefit the target group.

The training, selection, operation and supervision of social promoters hired through co-implementing enterprises is considered to be a highly effective management policy that has empowered the project's positive impact. Given the country's lack of prior experience, the success achieved demonstrates very effective human resources management by the PEU, as well as the existence of qualified personnel at the national and local levels who are highly qualified and whose potential was detected and developed thanks to the project.

Performance by partners was generally satisfactory. However, IFAD and the GoV did accept design deficiencies and failed to remedy them in a timely manner during implementation. The design errors that had the greatest impact on the project have to do with agricultural technical assistance and the design of the guarantee and rediscount system. These components were not properly reformulated, either during implementation or during the mid-term review mission (2002). The failures affected the level of disbursement of the IFAD loan and the achievement of greater positive impact on users.

Monitoring and evaluation functions performed unevenly, with some successes and a considerable number of failings. The M&E unit was responsible for producing dozens of quarterly progress reports, which are very complete and adequate for analysing each time period. Nevertheless, the reports were not consolidated, either for each year or on a cumulative basis, such that the project's overall progress on various aspects could be evaluated. No unique registration system was established for project users, leading to serious problems in identifying the number of users discriminated by type of service and avoiding duplication. The formulation of impact indicators, contracted out to a specialized enterprise, took a protracted period of time and an appreciable expense to produce inferior instruments. Much data and information was gathered in the field and from regional offices that was neither centralized nor centrally processed, impeding an overall appreciation of the project's impact. No baseline study was performed in a timely manner. An impact assessment on beneficiaries was conducted in 2003 on the basis of a well designed survey that does not however permit comparisons with the pre-project situation given the lack of a baseline study.

Main findings and lessons learned. Main findings and lessons learned from the PRODECOP experience can be summarized as follows:

  • Investing in development and citizen participation generates significant impact on building social capital and has an immediate effect on socioeconomic demand by the organized population. Training in the rights and responsibilities of citizens and their application through participation in decision-making on development planning and public investment promotes more inclusive, more sustainable democratization processes: Rural poor communities demonstrate a high degree of sensitivity and responsiveness to the presence and action of adequately trained development agents. This sensitivity increases in direct correlation to the prior absence of the public sector in these localities, and generates demands that transcend the local environment. Using appropriate instruments to promote and organize communities leads to a rapid and intense process of formulating participatory diagnostic assessments, setting priorities in terms of needs and programming petition actions and following up on initiatives. This points up the need to make adequate provision for resources and mechanisms to address increased demand. Mobilizing relatively idle local and regional resources is an appropriate response, as demonstrated by PRODECOP, but when such resources are scarce or nonexistent the inclusion of specific funds to meet such demand is always a necessity, as in the case of the FIC under this project. Otherwise, expectations may be frustrated, leading to a highly adverse impact on rural development.
  • It is important to promote and qualify local enterprises and train local professionals and technicians to provide training and technical assistance to rural poor communities. Encouraging the establishment and development of local professional and technical enterprises with a social awareness and the ability to support development processes broadens and qualifies the scope of social policy and allows for a better match between the population and the supply of state services and community development programmes and projects.
  • Given highly innovative project design features and/or major changes in the environment, it is necessary to establish closer and more timely oversight mechanisms for all those involved, to allow for adjustments and reformulations and avoid the underuse of originally allocated resources: PRODECOP failed to use a good part of the resources originally allocated by the GoV and IFAD as a result of inapplicable design features. This fact was known to the PEU and, through the cooperating institution, to IFAD, but corrective measures were not taken in a timely manner. Had a system of controls and more fluid communication been established, a timely change in direction could have enabled these resources to be used, for a much greater positive impact on the target population.
  • Projects with extensive geographical coverage including areas with major ecological, climatic, productive and cultural differences should build intervention modalities into their design that take such differences into account. The design of projects with such characteristics should not only recognize the differences but above all build in differential instruments, resources and time periods so as to contribute effectively to offsetting differences at start-up.
  • Promoting rural financial services based on a rural microfinance approach and comprehensive training for communities generates important benefits that transcend the financial sphere. PRODECOP shows that creating dozens of rural credit unions met a correctly identified need in a rural financial market that lacked depth, competitiveness and equity. Both the project design and practice showed that a component focusing on local financial intermediation can effect major changes in the rural financial landscape. Secondly, the decentralized nature of the functions of intermediary entities and the technical assistance system is crucial to the success of these proposals. Thirdly, at this level of development, the credit unions' legal underpinnings and internal controls are sufficient to continue offering services to members and a limited number of third parties. Only when the credit unions reach a higher rung of the financial market will it be necessary to introduce general regulations to protect non-member savers. Finally, the success achieved came about through five observed trends:
  • Synchronization : In all the rural credit unions created in the context of the project, a flow of resources has been systematized between low-income households with a supply of liquidity and others with a demand for liquidity. This flow takes place at positive real interest rates on both savings and loans. Symmetry of information and relations among participants have been crucial to this synchronization.
  • Inclusion : The rural credit unions promoted by the project tend to respond to four forces opposing marginalization: (a) Men and women make up credit union membership in proportions of 55%-45%. (b) The younger generation accounts for approximately one quarter of users. (c) A multi-ethnic membership has been observed in several rural credit unions. (d) Available evidence suggests that rural credit unions tend to include both the poor and the less poor.
  • Expansion : Growth patterns for rural credit unions are uneven and varied. Expansion takes place in parallel to the diversification of financial products, coordination among credit unions and access to external funding sources. In several cases, forms of cooperation have been observed that can be considered precursors to an eventual federation of credit unions.
     
  • Contribution to the social fabric: Although credit unions are not organizational vehicles designed for social change, they can be observed to occupy a real space in seeking out local arrangements and social dialogue. The credit unions developed positively as platforms for gender equity and as interlocutors for municipal management.
     
  • Ownership: Among the most positive features are an individual and collective sense of ownership and fulfilment on the part of rural credit union members. Feeling that one is a co-owner of an intermediary financial organization is a critical factor in the "ciudadanización" process and is likely to be irreversible.

If rural financial services remain circumscribed to small microfinance institutions, substantial demand for financing investments to transform the current technological profile of the rural poor will continue to be unmet. Designing adequate long-term financial systems to operate through local financial institutions remains a challenge. The relative immaturity of the credit unions is still apparent in the modest range of long-term financing services for productive units. Some consolidated rural credit unions are expected shortly to finance productive projects on a larger scale and for longer periods. The instruments proposed in the design (guarantee and rediscount funds) were neither timely nor appropriate. PRODECOP can orient, advise and expand the room for manoeuvre for the credit unions, but cannot force linkages with the banking sector. This does not mean that a higher volume of external financing would not have been welcome. Had the component been reformulated during the mid-term review (2002), a significant portion of the funding reserved for the rediscount and guarantee funds could have been allocated to an external financing fund. Such an adjustment in the PRODECOP architecture was sacrificed in the interests of a more ambitious and less achievable objective.  The changes made in Venezuela's rural financial landscape warrant shared reflection by the GoV and international agencies on how to build upon the rural credit union experience in the country. The credit unions have amply demonstrated that their model functions in the absence of an external incentive structure. The opportunity does not always present itself to expand, deepen and capitalize upon experience through another initiative to improve the performance of the rural financial market and foster linkages with other economic and social spheres. Nevertheless, such an opportunity would be well worth it.

Projects that include indigenous communities should have a strong cultural content, give priority to own technologies, traditional educational and health practices, and festive expressions invariably kept in the memories of elders. Project design and implementation should consider these aspects so that sufficient resources in terms of quantity and quality may be allocated to address specific cultural problems. Focusing action exclusively on combating material poverty when working with indigenous populations can reduce and/or mask the contribution by indigenous peoples to building more equitable societies living in harmony with ecosystems.

Following the gender approach effectively and sustainably requires not only that staff, co-implementing agencies and promoters are properly sensitized to gender but also that this be an integral part of planning field work, and of training materials and methodologies at all levels and the design of technical and advisory assistance provided to organizations. The work done by the project with co-implementing agencies, professionals, local technicians and promoters – both in raising awareness and training and in developing teaching materials and methodologies for the gender approach with rural populations– makes a contribution towards future interventions dealing with rural development themes generally.

Recommendations. These recommendations have been drawn up for the dual purpose of contributing to better project completion (December 2005 or the one or two year extension now being negotiated) and collaborating on the design of an eventual second phase.

Training for development and citizen participation component. It is recommended that: (i) Linkages between socio-community organizations and municipal and state bodies be deepened and made more dynamic. This implies developing training processes to support a future vision and medium-term planning; (ii) Training to build capacity among community organizations to participate in decision-making beyond the community or immediate interest group be strengthened; (iii) Resource allocation by the FIC (or other funding source) be implemented more dynamically and speedily to satisfy unmet demand for community and productive initiatives and projects still at the formulation and negotiation stage; (iv) Regional and area strategies be established for medium and long-term economic productive development geared to increasing incomes, with the economic productive organizations of greatest potential and interest; (v) The notion of co-financing be introduced for training and technical assistance services at the local level, on the part of organizations selected to ensure their sustainability following project completion.

Rural financial services and financial institutions component. Recommendations: (i) Gather and systematize the experiences of credit union participants to address the deficit in narrative documentation on creation and development of PRODECOP rural credit unions, representing the most important instance in the region; (ii) Promote the establishment of a modest database on rural credit unions for each state and municipality, to serve as a guide for any organization or agency that wishes to learn about and replicate the experience; (iii) Set up exchange and training opportunities for staff with national agencies specializing in advisory assistance and training for credit union executives. Topics such as arrears analysis and policy, application of differential interest rates, balancing portfolios with different financial income and expenses, interpretation of rates of return, creditworthiness and liquidity have not been addressed in support provided to credit unions and are critical to gain access to formal banking institutions; (iv) With respect to the development nature of the credit unions, they require guidance that will not impose outside criteria but interact with their own dynamic. Where feasible in terms of time and resources, PRODECOP may assist in drawing up more business plans than has hitherto been the case. In particular, a number of credit unions are preparing to grow in a coordinated manner (in the regions of the West). This may occur through trust operations among them or by means of a plan to create a federation of credit unions within asingle municipal territory. This calls for a multiyear strategy based on a consensus with individual credit unions and representative and competent leaders. Projects of this kind must be prepared for with an investment and institution-building plan. PRODECOP must prepare for this mandate, either under an extension or during an eventual second phase.

The gender approach. Recommendations: (i) Systematize the characteristics of the organizations promoted for their potential in terms of inclusion and adaptation to the needs and interests of men and women; (ii) Empower women's participation in economic productive organizations. It is not sufficient to carry out actions to raise women's awareness and self-esteem without also implementing affirmative action policies to foster economic ventures involving women.

Serving indigenous areas. It is recommended that: (i) Agreements and partnerships be entered into with research programmes and centres working on indigenous issues, in order to broaden the scope of project actions, strengthen and add value to interventions by technicians and promoters and foster an impact that will translate intervention on poverty into recovering and building culture; (ii) In planning work with the indigenous population, action on poverty be distinguished from action to value culture; (iii) Additional efforts be focused on recovering the cultural expressions of indigenous peoples which are central to promoting economic alternatives and improving living conditions.

Monitoring and evaluation component. It is recommended that: (i) A significant improvement be made, through new hiring and/or consulting services, in the quality of technical and centralized information processing resources in the PEU with respect to users, training events, extension activities, etc. (e.g. centralize a unique roster of beneficiaries), in order to gain a full and accurate view of the variables indicating actual project coverage throughout implementation. The aggregation and consolidation of relevant data (e.g. number of users, number of activities, number of initiatives) should be performed by periods relevant to the evaluation of results and impact (e.g. by year, cumulatively from the start-up date) rather than by implementation unit (e.g. by quarter). (ii) Steps be taken in regional offices and co-implementing agencies to reprocess information recorded in a decentralized manner in order to build up the M&E system as mentioned in the previous point; (iii) Resources be allocated to complete the systematization of experiences with rural credit unions, indigenous areas and other aspects mentioned in this report; (iv) An updated socioeconomic diagnostic assessment be prepared on the communities involved in PRODECOP, to serve as a baseline study for subsequent project phases.

Action at the governmental level: It is recommended that: (i) The GoV ensure the budgetary credits for use of the remaining IFAD loan proceeds, in the event an extension in the closing date is approved for Loan 427-VE, as well as local counterpart funding, in order to complete a series of pending activities; (ii) Continuity in PEU technical teams be assured until the project is complete, including the eventual transition to a second phase.


1/  The Interim Evaluation Mission was composed of Mr Ruy de Villalobos (Mission Leader), Ms María Elena Canedo (Organizations, Training, Gender and Indigenous Communities' Specialist) and Mr Hans Nüsselder (Rural Financial Services Specialist). Mr Paolo Silveri, IFAD Senior Evaluation Officer, joined the evaluation mission in its last week in the field.

2/ Transform into "citizens".

3/ See main text for further details.

 

LANGUAGES: English, Spanish

Southwest Anhui Integrated Agricultural Project (2006)

China  
September 2006

Completion Evaluation1

Introduction

The completion evaluation mission is conceived as an independent yet consultative exercise and designed as a service to the Southwest Anhui Integrated Agricultural Project (SWAIADP) as well as to IFAD's ongoing and future operations in China. The evaluation also served the purpose of accountability of IFAD to its governing bodies.

The Mission visited China from 23 June to 15 July 2005.

Methodologies utilised included field-level questionnaires, focus group discussions, interviews with key informants and direct observation of some 25 project sites. The evaluation adopted the prescribed framework of the IFAD Office of Evaluation.

SWAIADP was the thirteenth of seventeen IFAD projects in China, all of which have concentrated on agricultural development or rural credit or both. Six of these, including SWAIADP, have been co-funded by the World Food Programme (WFP). Total IFAD investment in China has now reached USD 400 million. The main thrusts of IFAD's Strategic Framework in the country are equitable access to natural resources, finances and markets, and the strengthening of grassroots institutions.

China has experienced a spectacular expansion of its economy in the 25 years since the initiation of market liberalisation and land reforms, during which time the average annual growth in GDP was nearly 10%. This has been achieved against radical structural shifts: from central planning to a market economy, from agriculture to manufacturing and services, from a closed to a globally integrated system.

SWAIADP was a USD 56 million project, financed by an IFAD loan of USD 26 million, counterpart funding of USD 22 million, local contributions valued at USD 5 million, and WFP food-for-training equivalent to USD 3 million. The bulk of the IFAD loan (68%) was directed to the provision of micro- credit. The project was declared effective in December 1997 and closed on 30 June 2004 after an extension of one year. Supervision and loan administration were carried out by UNOPS.

The main goal of the project was to reduce the chronic food shortages among half-a-million people - 123 400 households - living in the 34 poorest townships of the five poorest counties in the mountainous area of southwest Anhui province. An estimated 80% of households were categorised as poor or very poor at appraisal.

The project was implemented through seven components:

  • infrastructure (27% of base cost) – irrigation, roads, drinking water and flood control;
  • food crops (25%), development of arable land;
  • tree crops (26%), tea, mulberry, bamboo, chestnut;
  • livestock and fisheries (9%),
  • women's development (4%), training in literacy, health care and agricultural techniques;
  • social support (4%) and
  • institutional support (5%).

The provision of micro credit for poor households constituted the main instrument for implementation.

The Anhui Department of Agriculture was the chief implementing agency, under the control of the Ministry of Agriculture in Beijing.

Project Management Offices (PMOs) were established at the level of Province, Prefecture, County and Township, assisted by the relevant technical departments. The management and delivery of credit funds was to be handled by the existing Rural Credit Cooperatives (RCCs) at township level, the first instance in China of this innovative approach. A new institution, the Village Implementation Group (VIG), was created to promote and supervise village-level activities.

Implementation

The overall level of financial achievement at project closure stood at over 90%, with the outstanding 10% largely accountable to a faulty estimate of likely exchange rate fluctuations. In terms of physical achievements, around 16 000 ha of land was improved or put under new or rehabilitated cultivation, 320 km of roads were built or upgraded, 1 500 gravity tanks and wells were constructed for drinking water, 122 km of irrigation canals, 870 irrigation ponds and over 200 km of flood protection barriers were built, all of which matched or exceeded the physical targets planned at appraisal.

Credit management was handled jointly by PMOs and RCCs, with the PMOs normally bearing the credit risk and having the final say in loan approval. This arrangement was accepted by the People's Bank of China, which regulates cooperatives, and by the supervision missions. Nearly 100 000 households received credit under the project, or around 77% of all households in the project area. The largest proportion (46%) of the total loan amount was utilised for the cultivation of "economic trees", followed by short-term seasonal credit (25%). Overall recovery rates stand at around 85%.

Training programmes had a budget of nearly USD 2.5 million and included literacy and healthcare training specifically for women, technical training for farmers for the new crops, and loan management training for all households accessing loans. Some of the training materials and methods were found to be inappropriate for the illiterate and semi-literate. The WFP food-for-training acted as an effective incentive for attendance at the training sessions.

Relevance, Effectiveness and Efficiency

Project objectives were relevant to the causes of poverty in the project area, namely a shortage of cultivable land, low yields, lack of infrastructure, limited access to markets and poor levels of education, literacy and health, especially among women. SWAIADP was a ‘home-grown' project, first proposed by the Anhui Department of Agriculture and formulated largely by local consultants. However, the county PMOs argued with some cogency that the ‘menu' and extent of new crops might more usefully have been left to them.

Overall, SWAIADP was a highly effective and successful project in that it met or exceeded the planned targets. Supervision missions expressed reservations, however, regarding the targeting of the poorest and the effectiveness of the M&E system in capturing the impact of the project in this respect. The response of the PMOs was that in an area where virtually all households were poor, the example of more active and resourceful households would have the desired multiplicative effect. By project completion, 91% of targeted households had received loans, which appears to legitimise the PMO approach.

The prescribed monitoring and reporting systems were scrupulously followed but the Project Completion Report acknowledged some weaknesses: too much emphasis on quantitative data collection at the expense of data analysis, and difficulty in applying the results of M&E to the modification of village development plans, which accorded with the present Mission's verdict that there was a lack of qualitative impact monitoring and analysis.

The investment cost per beneficiary household was relatively low, at around USD 400, and the expected ERR, at 34.2% comfortably exceeds the appraisal figure of 22.6%. Management costs were far lower than comparable projects, a mere 2.8% of project expenditure. The major recurrent costs including salaries and transportation, as well as the provision and maintenance of buildings and offices, were met from existing government budgets. The commendable decision to maintain previous levels of government support in the project area meant that the operation of the SWAIADP constituted a genuine addition, rather than a partial replacement.

Some ‘collective' schemes were implemented, in which the planning and some of the inputs were provided by the implementing agencies. Plots are managed individually and the loans made to individual households, but the overall execution of the schemes was managed by officials. Instead of the mosaic of small plots with a variety of crops envisaged at appraisal, larger areas of monoculture have been established. The system was commended by UNOPS consultants and the Evaluation Mission was impressed by the efficient utilisation of scarce land resources.

Loan defaults of 15% indicate an area for concern. At appraisal, it was suggested that 80% recovery would be an acceptable rate initially but would be expected ultimately to rise to 98.5%, with the remaining 1.5% covered by the risk fund. The PMOs claim the recovery rate will gradually improve, but the only available sanction is the refusal of further loans.

Impact

The signs of increased prosperity were apparent everywhere: well-maintained metalled roads, renovated two-storey houses, well-stocked shops and markets. Rice, soybean, maize, mulberry, tea and chestnut plantations, wherever located, look uniformly healthy and productive. The total number of households reached by all activities represents 92% of households in the project area, comfortably exceeding the appraisal target. Nearly 5 000 ha of poor and unusable land has been improved by levelling, soil replacement and drainage, with areas of waste land transformed into highly productive orchards.

Per capita income and grain availability have been the main impact indicators utilised by the project. The average increases by county were 36.8% for income and 13.3% for grain availability between 1997 and 2003. From one mu2 of tea, a household can generate an income of Y 2 000 p.a., and from one mu of mulberry Y 10 000 from four annual silkworm harvests. Chestnuts were a successful option for households with a shortage of labour capacity, since the trees need only minimal tending.

Determining the extent of food insecurity is hampered by the lack of recent official records, which were abandoned as food became freely available at markets. Income and grain availability figures deal in overall averages, which might increase due to large gains made by the better-off. Much emphasis has therefore been placed by supervision missions on reporting the ‘graduation' of households in terms of the three categories of poverty. According to PCR figures, the percentage of households in the ‘poorest' category has fallen from 15.1% in 1997 to 2.6% in 2003, while that of the better-off households has risen with increasing rapidity.

Overall grain availability rose from 135 kg per person per year to 281 kg during project life according to PCR figures, 280 kg being the approximate yearly requirement in China for an adult with an average workload. There is of course inequality in terms of food availability, but no household met by the Mission, even the poorest, was unable to afford daily meals. Further, household diets were richer in protein and fresh vegetables.

SWAIADP loans were the first experience of borrowing for the majority of households. Project officials stated that their greatest problem in the early days was the suspicion and hostility among all but the most progressive households to the notion of borrowing. In all villages visited by the Mission, project-funded loans were found to be familiar and sought after, and the disciplines of loan management and repayment had become part of everyday life.

Particular importance was given to women signing for loans for mulberry cultivation, silkworm raising, animal husbandry, primary tea processing and other IGAs. This may not have been an appropriate emphasis in the case of southwest Anhui. Questions on this subject provoked an emphatic response from men and women alike: it was immaterial who signed the loan. Many loans signed by husbands were in fact utilised by women.

Rural Credit Cooperatives have benefited from the project credit operation, mostly without risk to themselves. Their deposits have increased, their clientele base has been broadened and their capacities in terms of loan approval and recovery have been enhanced. They are now aware of the potential of small loans made to poor people, provided the costs are covered by the interest charged.

The question mark hanging over the RCCs is whether they now have the capacity to carry out credit delivery, loan approval and recovery and to bear the credit risk in line with the original design. The feeling among the PMOs is clearly that the first responsibility of non-governmental cooperative institutions such as the RCCs will be to their members, and their primary aim will be profitability. IFAD is now testing this in the ongoing Rural Finance Programme.

The Mission did not see a single trace of deforestation or soil erosion in all areas visited. An extraordinary degree of care is taken in the terracing and cultivation of the slopes. Soybeans are grown on terrace verges for maximum utilisation of the soil and the prevention of erosion. In fruit orchards, mulch is commonly used for moisture retention. All maize, mulberry and tea crops are grown at reasonable density to maximize benefits and minimize surface run-off water.

Protected ‘niches' have been created for farmers to develop their land by the ‘bombing' of hilltops and mountains with the seed of pinus species. Field-to-forest conversion is subsidised.

Rural women have been key participants in project activities. The PCR assigns 40% of loans to women, apparently a mix of signature and utilisation. Their income from mulberry and silkworms, from tea cultivation and primary processing, from chestnuts and improved livestock breeds average at more than Y 2 000 p.a. Health awareness and disease prevention have improved. In terms of workload, women told the Mission they are busier than before, and content that it should be so in view of their increased income.

Greater awareness of disease, hygiene and nutrition was reported by women participants in healthcare classes. The declining incidence of gynaecological diseases is indicated by township data and confirmed by doctors, and staff of the Women's Federation. The consumption of meat, eggs, fresh vegetables and edible oils has greatly increased according to household interviews.

Lessons learned from SWAIADP by government departments include: (i) the advantages of ‘loan money' over ‘grant money' through the disciplines of repayment and the emphasis on sustainability; (ii) the usefulness of PRA techniques in needs assessment; (iii) the importance of compiling gender disaggregated data. SWAIADP has also boosted agricultural extension in terms of the range of services, with ‘strategically important' crops such as rice, wheat and maize being joined by cash crops including tea, mulberry, bamboo, chestnuts, gingko, kiwi, pears, peaches, plums, melons and grapes.

SWAIADP has built-in advantages in terms of sustainability: (i) the operation of the revolving credit fund for a further 15-20 years; (ii) the Government commitment to continued support through Food-for-work, increased funds for farmer training, further investments in rural infrastructure, and the provision of market information; (iii) agro-industrial initiatives (notably in Huoshan county) including primary and secondary processing and marketing, for tea, silk and bamboo; (iv) the survival of experienced PMOs, albeit reduced in size; and (v) above all, an environment well-protected against soil erosion and deforestation.

Innovations of the project include: the successful utilisation of the RCCs, in a somewhat restricted role, in the management of microcredit; the WFP food-for-training (as opposed to the well-tested food-for-work); the successful twinning of household choice and village-level planning regarding economic trees; the creation of the project-specific VIGs, which are now regarded as permanent institutions in villages.

Performance of partners

IFAD can be commended for an intelligent project design, the boldness of the new approach to credit delivery and the strong personal support of the then Country Portfolio Manager, mentioned by staff at all levels. It can be criticised for the lack of an initial survey of RCC capacities, which might have given advance warning of the problems in that sphere. In addition, the counties commonly requested more autonomy concerning the stipulations as to areas and outputs of new or expanded cultivation and the lifting of the Y 3 500 ceiling for loans.

UNOPS fielded six supervision missions, the first two of which were longer in the field, visited all five counties and produced more detailed and objective reports. The PMOs recommended a greater technical expertise in the membership of Supervision Missions, but generally appreciated the collaboration. The 2001 mid-term review was critical regarding the targeting of the poorest, inappropriate training for illiterate farmers and inadequate support for women's IGAs.

The provincial level PMO was the nerve-centre of the project. The contacts between administrative levels are frequent and the managerial ethos is marked by enthusiasm and frankness. PMO leaders at county level have a strong sense of ownership of the project. The Mission visited 12 out of 34 townships, where informative reports were presented and even searching questions answered without confusion or hesitation. As for the VIGs, the Mission's initial doubts as to whether the necessary capacities existed at this level proved unfounded.

As co-financiers, the WFP took a close interest in the project and its supervision. The WFP office in Beijing provided IFAD with in-country representation. The management of the food aid was effectively organised and losses in transit kept to a minimal level.

Recommendations

Pilot schemes for the latest generation of microfinance approaches and/or the proposed Village Development Funds might be appropriate ways of building on IFAD's investments in Anhui province.

The experience of SWAIADP indicates that with time the ‘model farmer' approach to targeting was capable of reaching the poorest households. Where the necessary degree of homogeneity and social cohesion exists in villages, inclusion of all households is to be preferred. Microcredit lent to the poorest households should have more favourable conditions attached, such as longer terms and a greater number of instalments.

Saturation in the markets for raw silk and tea may pose a risk to the economic sustainability of project interventions. The diversification of income must be seen as an ongoing process sensitive to market needs and changes and as requiring further support, notably through appropriate training programmes for agricultural extension officers and ready access to market information for producers.

The training offered to women has not always been appropriate to their educational level and needs. Recommendations include: the replacement of traditional literacy training by simplified functional training; technical and credit training specifically for women; training materials with more illustration, a less densely presented text and a larger font; the training of trainers for more effective healthcare programmes; shorter and more frequent sessions; and smaller classes.

Data on the incidence of disease is not effectively utilised. Healthcare workers should submit the results of medical check-ups to the township PMOs who, with the assistance of the hospitals, can investigate the causes of high rates of disease in particular villages and tailor the healthcare training accordingly.

The aim of 100% loan recovery should be re-emphasised and ways of achieving it investigated and discussed at county and township level.

The baseline survey was over-complicated. The questionnaire ran to many pages and included superfluous data. Perhaps as a corollary of this, the sample of households (250 in all) was too small to be representative. For comparable IFAD projects in the future, a shorter questionnaire and a larger sample is recommended.

The approval of the AWPB required a cumbersome process, passing through three levels of aggregation and approval prior to the donor approval process involving IFAD, WFP and UNOPS. Simultaneous in-country approval of the AWPBs for ongoing projects should be mooted to ensure a more participatory consideration of the AWPB by all parties simultaneously, as well as the abbreviation of the process.



1/ The completion evaluation mission was composed of: Mr Roger Norman, team leader; Mr Pham Quang Hoa, agriculturalist; Mr Yusaf Mahmood, management and institutions specialist; Ms Han Zheng, gender and targeting specialist.  Mr Flemming Nichols, then OE Evaluation Officer, was overall responsible for this evaluation and joined the mission in its final week in the field.

2/There are 15 mu to a hectare.

 

LANGUAGES: English

Upper West Agricultural Development Project (2006)

Ghana  
September 2006

Interim Evaluation

In accordance with the Evaluation Policy of the International Fund for Agricultural Development (IFAD), the Office of Evaluation (OE) conducted an Interim Evaluation of the Upper West Agricultural Development Project (UWADEP) in Ghana in May-June 2005, given the interest of both the Government of Ghana and IFAD's Western and Central Africa Division to proceed with further investments in the area. This evaluation adopts the standardized IFAD methodological framework for project evaluations.

IFAD projects in Northern Ghana. Northern Ghana consists of three regions, Upper East (UER), Upper West (UWR) and the Northern Region (NR). IFAD has projects in each of them: Land Conservation and Smallholder Rehabilitation Project (LACOSREP), UWADEP, and Northern Region Poverty Reduction Programme (NORPREP) for the Northern Region. UWADEP originated from a FAO General Identification mission in 1993, was appraised by an IFAD team in mid-1995 and modelled on the example of the existing LACOSREP I. UWADEP was approved by the IFAD Executive Board in September 1995, became effective in March 1996 and closed in December 2004. The total project cost is USD 11.3 million, out of which IFAD provided a loan for USD 10.0 million. As of August 2004, the disbursement rate reached 94.64 % of the total loan amount. IFAD is the only international financier of the project, which was supervised by the United Nations Office for Project Services (UNOPS) for most of its implementation.

Macro-economic and poverty indicators. Located in West Africa, Ghana has an estimated population of 20.5 million, of which 63% rural. The structure of the economy is characterised by a large (in relative terms) services sector (42% of the total GDP, compared to 34% for agriculture and 24% for industry). It has an annual GDP per capita of USD 304 and GDP growth has averaged 1.8% in the last ten years (i.e., below population growth) although this has accelerated recently. Agriculture continues to be the mainstay of the economy, employing about 60% of the labour force. Ghana is classified as 131st out of 175 countries, by the UNDP Human Development Index (2004). The percentage of households below USD 1 per day has been estimated at 44.8%, and the percentage of poor households according to a national poverty line at nearly 40% (World Bank, World Development Indicators, 2004).

Agricultural policy. Agriculture contributes to ensuring food security, provides raw materials for local industries, generates foreign exchange, and provides employment and incomes for most of the population. Agriculturally-dependent rural households (72% of the population) form the largest potential domestic market for output from other sectors of the economy. Recent agricultural policy in Ghana is reflected in the following key documents: (i) Medium-Term Agriculture Development Programme (MTDP) (1991-2000); (ii) Accelerated Agricultural Growth and Development Strategy (AAGDS); (iii) the Food and Agricultural Sector Development Policy (FASDEP) (2002); and (iv) the Ghana Poverty Reduction Strategy (GPRS) (2002-2004), currently under revision.

The Ghana Poverty Reduction Strategy (GPRS). GPRS (2003) was revised in 2005 to focus on the identification of vulnerable groups and possible risk-reduction strategies. It recognises that rural farmers and fishermen are particularly at risk and specifically mentions Northern Ghana as a locus of perennial food deficits. Women are identified as particularly discriminated in this context and instruments to promote gender equality are emphasised. The document also notes the importance of environmental factors in increasing vulnerability. However, it does not put forward concrete measures to reduce risk in agriculture.

The rationale for UWADEP was summarised at appraisal as:

  • strong demand for dam rehabilitation in rural communities;

  • the potential for Water User Associations (WUAs) to be sustained and assure food security in the region; and

  • build on existing credit experience to establish effective mechanisms for rural financial institutions.

Strategic thrust. UWADEP was intended to improve food security and increase the income of smallholders. Its components include:

  • capacity building to strengthen project delivery and management skills of key implementing agencies;

  • water resources development (rehabilitation of dams, formation and support to WUAs, catchment area protection, and demonstration/promotion of manually-operated tube wells);

  • agricultural development, including farmer training and demonstrations, support to technology generation and research/studies, marketing and processing, and livestock development;

  • promotion of income-generating activities through the supply of rural financial services; and

  • rural infrastructure comprising rural road rehabilitation and the construction of hand-dug wells for drinking water and latrines.

Project area. The UWR is situated in the northwest corner of Ghana, with an estimated total population of 580 000, of which about 90% is rural. The average population density is 29.8 persons/km2, about one fourth that of the UER. UWR is one of the poorest regions of Ghana, with an annual per capita GDP of USD 170 and on most social indicators, the most neglected region of the country. Infant mortality, seasonal hunger, cash incomes, school attendance, transport networks are weaker in UWR than other regions. Despite this, UWR has benefited from very few targeted development projects, and until recent years, not many NGOs were operating there. UWR has abundant land both for crops and livestock and much lower population density, but access to markets and off-farm opportunities is constrained by poorly maintained feeder roads and lack of transportation services.

Components of UWADEP at appraisal were as follows: (i) agriculture (33% of total base costs); (ii) water resources (17%); (iii) rural roads (16%); (iv) credit (22%); (v) community and women's development (6%); and (vi) project support unit (7%). The target group is poor farmers, which represent up to 80% of the population of UWR, some 20 000 households according to the 1995 appraisal. Overall responsibility of the project was with MoFA, with the Chief Director responsible for policy direction and the provision of counterpart funds at regional level. A Project Support Unit (PSU) was to be established within the Regional Office of Agriculture to assure project programming, prepare work programmes and budgets. Community mobilisation was to be the responsibility of an officer seconded to the PSU to establish linkages with the Community Development Department and Women In Agricultural Development, to assure the emphasis on gender.

Implementation

Rural credit for income-generating activities. The project reached 5 805 beneficiaries in 379 groups for a total of circa ¢ 5.6 billion, equivalent to USD 640 000 (about 60% of target). Even though women constituted 56% of this number they received only 47% of total loan amount. The largest portion (64%) of the disbursements went to farming which is dominated by the males. Loans for income-generating activities such as trading and food processing, reserved for women, followed with only 24%.

Dams, irrigation, water and roads. After the end of extensions, the project finally closed, leaving irrigation infrastructure (dams and canals) incomplete on several sites. The draft Project Completion Report (PCR) states that 41.5 ha are under dry season cropping and 154 ha available for irrigation against an Appraisal Report (AR) target of 220 ha (70%). The Ghana Irrigation Development Authority (GIDA) Technical Review on completion (June 2005) mentions the same figure of 154 ha, but cautions that certain areas are not under the command from the canals, which means that hand watering or pumping has to be used. Farmers still largely depend on hand-dug wells, as was the case before the project rehabilitations took place. The evaluation team could find only 23 ha. of additional irrigable area resulting from the project (figures obtained from physical observations and cross-checked with WUA secretaries/chairmen in each case). Prescribed sanctions were not applied to defaulting contractors in several instances by PSU. Generally, no laboratory investigations for quality assessments of the work were conducted. This is a major problem, as many structures suffer from poor quality, and thus uncertain sustainability. After the two extensions, the project finally closed in 2004, leaving irrigation infrastructure incomplete on several projects. Through the Ministry of Food and Agriculture (MoFA), additional funds were provided from Food and Agriculture Budget Support (FABS) to cover the outstanding works. The AR mentions involvement and training of WUAs, as well as specifically-targeted interventions for women. In the AR, 40 hand-dug wells were planned at the dam sites to provide safe drinking water to participating communities. About 35 of these were sunk - 12 in 2004 alone, by an NGO (PRONET, Wa). The target for feeder road infrastructure in the AR was 140 km and this has generally been exceeded.

Agricultural extension and seed production. Prior to UWADEP, the Seed Growers Association (SGA) in the region had only 12 growers who had previously produced seed, mostly maize, under contract to seed supply firms. Considerable progress was made prior to decentralisation, with the number of growers increasing from 12 to 60 by 1999. Numbers have since reduced, with 30 successfully producing seed in 2004. On-farm demonstrations. Demonstrations of the application of single super phosphate (SSP) to groundnut were promoted throughout the project life. The project worked with a total of 3 600 farmers in 300 groups, 75% of the target of 4 800 farmers in 400 groups. Support and promotion of animal traction. The implementation progress of the animal traction component has been very modest at field level, with only 130 carts, 65 ploughs and 65 ridgers made available to 260 individuals, training conducted and refurbishment of one blacksmith's workshop at Tarsaw.

On-farm adaptive research. The on-farm adaptive research component had two major activities: infrastructural support to the establishment of a research station in the UWR and implementation of research in support of improved crop production systems for the region. A permanent base was established for the Savannah Agricultural Research Institute (SARI) at Dokpong Agricultural Station at Wa, with the construction of an office block with laboratory facilities, accommodation for ten staff and a large conference hall. Initially SARI conducted a wide range of trials attempting to cover the issues identified during the initial project planning workshop held in 1995. Research subsequently focussed on two main areas; improvement of soil fertility, and identification of improved varieties of the major legume and cereal crops. Introduction of improved varieties of cowpea, soybean, sorghum, maize and rice have generated much enthusiasm among farmers, however access to certified seed remains a problem for small-scale farmers.

Upgrading of local livestock. During 1998 and 1999, 242 improved rams and 90 bucks were distributed to farmers in the five districts. High levels of mortality led to concerns over the approach and the programme was suspended for two years. It was restarted in 2002, with a refocus on beneficiaries with sufficient resources to be able to afford improved housing, feed and recommended health measures. Some 219 (50%) offspring of the improved rams and 123 offspring of improved bucks have been recovered for redistribution. Poultry and guinea-fowl. Between 1998 and 2004 some 13 640 keets were imported from Belgium. The average survival rate for the first two batches was 84% (M&E unit, 2002). Keets were sold from the 1998 batch with 24% subsidy, but from later batches at full cost recovery. Beneficiaries report that the imported guinea fowls were robust and about twice the size of local birds. They fetch ¢35,000 as compared to about ¢20,000 for local birds. Training and support to Community Livestock Workers (CLWs). The CLW scheme recruited and trained 150 volunteers and provided them with basic livestock kits to help improve livestock health and nutrition within project groups and in time, within their whole community. Groups report significant reductions in mortality rates with the instigation of recommended management and health care.

Performance

Relevance. The overall objective of UWADEP was to empower rural populations living in poverty to access improved technology services and credit. There is no doubt that the overall and specific objectives are relevant to these rural communities which depend almost entirely on agriculture. Ghana has signed up to various international undertakings to reduce poverty and UWADEP has this as its direct focus. Despite its commitment to donor strategies on poverty reduction, the government of Ghana has yet to make available additional funding for the development of these regions, which makes IFAD's approach all the more relevant. However, allocation of resources within sub-components between infrastructure development (SARI research station, animal traction centre and Livestock holding centre), training, demonstrations and practical interventions had limited relevance. As an example, the focus on training within the animal traction component seems misplaced in the context of working with groups of farmers who already own bullocks and donkeys.

Effectiveness has been low for the rural finance component: transaction costs are high for both banks and clients and credit discipline has been weak, with the exception of Sonzelle Rural Bank (RB). In the agricultural development component, AT interventions were limited in spite of high farmer demand. Areas of investigation in adaptive research have been limited, surprisingly excluding soil and water conservation. Research has ignored traditional farmers' crops such as yams (without practically any linkage with IFAD's Roots and Tubers Programme) and the spontaneous development of hand-watered gardens along rivers. Improvement of small ruminant and poultry/guinea fowl breeds has proved popular, in spite of an outgrower scheme that prioritised the better off, with delays to the servicing of poor farmers. Supervision of dam construction has been very weak and characterised by intra-government disputes.

Efficiency has been explored for irrigation, through the comparison of construction costs per ha between the two phases of LACOSREP and with cost ranges of other organizations (cost-effectiveness analysis). In the case of the rural finance component, productivity and efficiency (as per Consultative Group to Assist the Poor (CGAP) definitions), are compared with regional benchmarks. Unit costs for irrigation and rural finance compare favourably with their peer observations (in the case of irrigation this is partly due to the devaluation of the cedi). It is however important to contrast low costs with limited achievements (for example both in terms of delays and poor infrastructure quality).

Impact

Methods. Findings are the result of triangulation between multiple sources: (i) quantitative surveys of beneficiaries and non-beneficiaries; (ii) a qualitative survey of five dam sites; (iii) an ad hoc survey of non-beneficiaries; (iv) participants' observations as recorded by mission members; and (v) secondary data.

Outreach and impact. Physical and financial assets of farmers assisted by the project are rising, but there is a general increase (i.e., also for non-project farmers) in access to material assets due to remittances. The choice of training topics has not always been appropriate, thus the impact on human capital was limited and limited attention has been devoted to human health hazards under the water management component, in spite of the potential risks of water-borne diseases. A very important and originally unplanned development took place when under-privileged groups (blind, disabled and single-mother) were the object of the project's assistance in Karni dam site. This happened also thanks to preparatory work carried out by an NGO. Attention to gender issues has been limited throughout the components. The present implementation does not ensure that benefits spread rapidly to the resource poor and women. If there is one message that emerged clearly from the field visits, it is that training, research and sensitisation has been over-emphasised to the detriment of assisting farmers with practical requests.

Innovation, scaling-up and replicability. Few elements of UWADEP can presently be considered an unalloyed success and therefore the desirability of replicating them in their present status is questionable. Clearly more dams would be desirable; one option is to use the NGO sector to deliver more modern dams. The introduction of improved breeds and seeds has proven partially successful, but could have achieved a wider diffusion if they had been more responsive to farmers' requests. Improved breeds of small ruminants and fowl have been successful in increasing incomes for a range of farmers and would seem suitable for replication.

Sustainability has been constrained in the rural finance component by below-market interest rates to final borrowers and poor credit discipline. Sustainability of the SGA is not assured because of the failure to link the group effectively with inventory credit and missing links in the seed chain between small farmer's demand and supply. In the irrigation component, the main issues are: completion of irrigation infrastructure through FABS, which, given the pace of project implementation during UWADEP must be of concern to communities, and quality of works (which is manifestly poor). Few WUAs have been in operation long enough to judge their sustainability, but in many cases these social groupings are robust because they have existed in a different form prior to UWADEP, managing the hand-irrigated land below the dams, sometimes for decades.

Performance of partners

IFAD. Two issues are critical in assessing the performance of IFAD; the acceptance of an appraisal document with features that were clearly inappropriate for UWR and an inadequate response to design and implementation problems which surfaced during the course of UWADEP. For all the participatory rhetoric, the design of UWADEP was top-down and failed to address the concerns of a high proportion of its target group. Design decisions in irrigation infrastructure should generally be flexible and appropriate to the actual environment, but those in the AR were too rigid, giving very little room for changes to suit individual sites.

UNOPS supervision seems to have been less accurate than in LACOSREP II (see the related evaluation for comparisons), the sister project in UER, in that many of the problems and inflated claims on achievements of UWADEP were not picked up. Given the time and resource costs of visiting individual communities to establish the exact implementation status, this may well be considered beyond the present resources and contractual arrangements between IFAD and UNOPS.

Government and its agencies. A characteristic feature of the PSU was its use of available MoFA staff, even when sector specialists were not present. Thus the PSU had no staff qualified to supervise infrastructural work, or to monitor and encourage gender sensitivity. There was no specialised staff member in the PSU dealing with credit. Changes in project management to introduce dynamic individuals took place late. The failure to engage with NGOs and the use of Agricultural Extension Agents (AEAs) to perform their tasks ‘after hours' is doubtful practice at best and hardly develops the grassroots and innovative approaches expected by IFAD. Project documentation was lamentable, and even after considerable work, the numerical data on implementation presented in this report must be treated with caution. The decision to use GIDA as a sole consultant for the irrigation infrastructure on the project was a very questionable one, causing many problems in the quality of project execution, as well as delays in implementation.

Overall assessment

The overall impact of UWADEP must be considered modest. The number of dams functional by the closure of the project was small and the irrigated area limited. Many of the dams have engineering and technical problems which bespeaks a lack of supervision. As a consequence it is hard to evaluate the increase in social capital and the strength of WUAs, because they have had a limited chance to operate. Much of the irrigated area claimed by UWADEP turned out to be well-based gardens that were in operation prior to the project and are of greatest use to the younger and stronger members of the community. Appraisal targets for women's involvement were almost never met. The majority of resources were spent on capital goods and central infrastructure with very little visible return and the agricultural components were unresponsive to the needs of actual farmers. In the case of animal traction, for example, expenditure on training and buildings far exceeded sums spent on getting implements into the community, despite the strongly voiced requests of those communities. Partnership with NGOs was weak. Credit was taken up and clearly valued by its recipients, but little emphasis on supporting rural finance institutions and cumbersome procedures meant that its impact was far less than its potential; moreover, most of the banks appear to be simply withdrawing rather than continuing with their clients. A fundamental problem was the copying of many features of LACOSREP without due consideration for the different ecological, economic, social and demographic characteristics of UWR, a design shortcoming for which IFAD must take some responsibility. However, many problems also arose from weak supervision and the MTR process. It would not be desirable to replicate or extend UWADEP in its present form. Further interventions in the area, in principle justified by the severity of poverty, require major changes in the project design.

Insights and recommendations

Insights

Further investment? If IFAD is to consider further investment in UWR, then the evaluation of UWADEP undoubtedly suggests major rethinking of structural elements in project design. A key problem is that projects with so many components, but no clear integrative strategy, are open to activities being carried out with no linkages, with consequently high management costs. The failure to link credit with other technical innovations suggests the added value that could be gained from a more coherent approach. The significance of delivering infrastructure in a timely manner cannot be overemphasised. No amount of training and sensitisation will compensate for this fundamental lacuna. Project design should consider sequencing much more carefully. Despite efforts to introduce greater clarity into contracting arrangements, the reality has been almost the reverse.

What type of project? In Ghana, in some sectors such as health and education, donors have started to fund multi-donor budget support initiatives. Although the pressure is not yet strong in the agriculture and water sector, the question is whether area development interventions deserve further funding. UWADEP illustrates that there is a very significant need on the ground for quite conventional projects that deliver services and inputs to impoverished farmers. It is yet to be demonstrated empirically that other formats can deliver the same benefits. Should IFAD decide to fund future investments, clearly a priority would be to improve the project concept and design. Another priority for IFAD and the Government would be to identify strategic partners. With its own resources only, IFAD's coverage will be inevitably limited.

Recommendations

Project design and partnership with innovators. UWADEP developed from the models of URADEP and LACOSREP, rather than being designed to address the specific situation of the UWR. This has been problematic, because its interventions are not necessarily those most valued by communities. Key areas of agricultural production such as yams and tree-crops were not considered, although these provide cash income and also help communities bridge the food gap experienced by cereal growers as well as reducing migration. In addition, the components were poorly integrated at the design level and continued largely to be managed separately, with consequently high transaction costs. NGOs are not necessarily better than their official counterparts, but major NGOs in UWR are rapidly developing capacity to cover many of the same areas as government-based extension. A future project design that does not recognise these realities would be highly problematic. There is a strong need to learn from "good practices" by other organisations, including particularly NGOs in UER and UWR. These practices should be documented and discussed not only at the regional level but also in national fora, perhaps taking the opportunity of the donor's coordination group in the agricultural sector.

Dams are one strategy in a basket of options for improving household incomes. Other options have been discussed in the report and are presented later in this section.

Improve implementation support and monitoring and evaluation (M&E). The MTR was delivered late, but even so, it is very evident that most of its recommendations were not taken into consideration by the PSU. The PSU needs to be more proactive and so do UNOPS and IFAD to ensure follow-up. UWADEP appears at first sight to have a comprehensive record of its impact. However, many of the figures recorded turned out to be inaccurate. Record-keeping in UWADEP collaborating institutions, notably the banks, has also been weak.

Agriculture. Despite the rhetoric of participation and ‘demand-led', on-farm research as practised by the project is surprisingly similar to the testing of a package of improved technologies under farmer conditions. The almost universal use of organic manure by farmers, limited practice of composting and difficulty in affording and accessing fertilisers was not reflected in trial designs. More attention needs to be paid to women's crops (such as vegetables) and cropping systems, where there is little use of chemical fertilisers. Farmers' requirements with regard to animal traction tools (for example ridge weeders and robust furrow weeders) should be included in the selection of tools made available. Institutional arrangements for supply of tools at local level should be explored with NGOs. Exclusive focus on demonstrations of an application of SSP to groundnut led to limited results because of the unavailability of SSP and the farmers' inability to afford fertiliser inputs following withdrawal of the project. The introduction of 50% Sahelian improved stock would enable a greater number of farmers to take advantage of the intervention more quickly. As the 100% improved stocks do not survive for long under the traditional husbandry system, the approach would be more popular with the majority of farmers.

Additional areas for inclusion. Despite their importance in the local economy, their expanding production and their capacity to bridge the ‘hungry season' and the existence of the IFAD Roots and Tubers Programme, tuber crops have been virtually ignored. Yam and frafra potatoes have a good local market and in the case of yam, very good external markets. Agroforestry systems may be particularly suited to the region and help introduce permanent cropping alongside annual cropping to improve both production and income levels. Indigenous agroforestry systems based on the cultivation of yams and vegetables below certain trees, such as locust (Parkia biglobosa), neem (Azadirachata indica) and Acacia (Acacia albida) are a local response to soil fertility constraints and are worth investigating for wider application.

Rural finance. Financial sustainability depends very much on the participating banks profitability, which is based on financial income, operational cost and loan loss provision. As in several projects built around the ‘traditional' view, rural financial institutions in UWADEP have been seen as a mere conduit to provide an input (credit) to farmers. Institutional strengthening of financial institutions has not been seen as a priority. In spite of this, Sonzelle RB has been able to enforce good credit discipline and attain operational (although not financial) self-sufficiency. It is the focus on sustainable institutions that should characterise future interventions, either through a project component or a dedicated programme (in which case the issue of dovetailing with agricultural interventions in other projects becomes crucial). There is a wide range of credit facilities available when the District Assembly Fund, international NGO and local NGO activities are considered. Specifically, a large number of organisations are involved in inventory credit and credit for agricultural inputs and all offer credit at different rates, i.e., there is predatory competition in a poorly regulated environment. Without region-wide co-ordination, market led microfinance cannot succeed, as it will be out-competed by politically and socially-motivated interventions.

Water and infrastructure. Civil works on dams and irrigation infrastructure were the major cause of delays in project implementation. A quite different approach should be considered in future, with checks on quality mandatory at every step of implementation. Opening the consulting to a wide range of professional companies should be non-negotiable. Services should be segregated into the phases according to the project implementation schedule and contracts signed separately, whereby the contract for downstream works is not automatic, but subject to satisfactory performance in the first phase, if the same consultant is selected to undertake the assignment. For projects to be completed within the scheduled completion period, within the budget and to specification, stakeholders, comprising the project staff, consultants, DA, and the WUA should take active part in monitoring construction works. Finally, it should be recognised that irrigation infrastructure needs maintenance, some of which is beyond the capacity of the WUAs. This should be reflected by realistic budgeting.

 

LANGUAGES: English

North Eastern Region Community Resource Management Project for Upland Areas (2006)

India  
April 2006

Interim Evaluation1

The independent interim evaluation of the North Eastern Region Community Resource Management Project for Upland Areas (NERCRMP) was undertaken as a standard procedure in the process of preparation of a possible follow-up phase. The evaluation objectives were: to assess and document the results, impact and performance of the Project and of the partners; and to produce the basis for an agreement at completion point (ACP).  The findings, conclusions and recommendations in this report emanate from 17 days of field work, covering all six participating districts in the three states of Assam, Manipur and Maghalaya, 50 villages and intensive interaction with over 1 500 farmers, drawn from self-help groups (SHGs) and natural resource management groups (NaRMGs), with near equal representation of men and women.  The report takes into account the IFAD Strategic Framework, PI Regional Strategy and COSOPS of 1999 and 2001, and the development strategies and views of the Government of India, the North Eastern Council (NEC) and the Project Management

The Project was designed in the period 1994 to 1997. The main justification was the near total failure of previous top-down, culturally inept development initiatives; economic stagnation and resultant chronic poverty; inability of the traditional jhum system of shifting cultivation to cope with increasing population and decreasing fertility, with consequent threat to environment and biodiversity; and the general air of distrust and disillusion about Government intervention. Given this background, the primary thrust of the strategy was to introduce a development approach that was technically appropriate, culturally sensitive and institutionally effective. The aim at appraisal was to cover 460 villages in two phases of three then four years. Project area villages were to be selected to prioritise the target group criteria of: dependence on jhum; small farm acreage; rainfed cultivation; and prevalence of disadvantaged, vulnerable families. Specific emphasis was placed on empowerment of women; water catchment integrity; and pro-poor, differential benefit entitlement, to augment traditional tribal poverty alleviation customs and practices.

The goal of the Project was to improve the livelihood of vulnerable groups in a sustainable manner through improved management of their natural resource base.  Further specific objectives highlighted: local peoples, and women's participation; community organisation; environmental know-how; savings and thrift; and access to basic services and infrastructure. The original components were: Capacity Building of Communities and Participating Agencies; Economic Livelihood Activities (farm-based and non-farm); Community-based Biodiversity Conservation; Social Sector Activities; Village Roads and Rural Electrification; and Project Management. Following Mid-term Review in June 2002, the components were structured to comprise: Institutional Support (~7% of funding); Village Development Fund (VDF), including Livelihoods (~52%), Social Sector (~5%), and Village Infrastructure (~20%); Natural Resource Management (~4%); and Project Management (~11%).    

The implementing partners are Government of India, under the aegis of the North East Council (NEC), embracing the three State Governments and the Regional and District Societies, formed by the Project to address the inadequacy of existing development management capacity. These societies regroup representatives of NEC, relevant central and state line departments, parastatal agencies, District Administrations, participating communities, NGO partners and civil society; at the village level, the SHGs and NaRMGs, NGO facilitators and supervisors, district and block line departments and private sector service providers. Finally, the executive management rested with the Regional Programme Support Unit (RPSU) and six District Support Teams (DSTs), each with a District Project Manager and with a core cadre of six to eight officers.

The most notable policy and institutional change made by the Project was the successful establishment of village and community groups and their NGO service providers – all the more so that evidence suggests that emerging that future district development policy is likely to be aligned with these principles. The major design change took place at MTR: an increase of the target coverage from 460 to 1 000 villages; and introduction of the Village Development Fund.

As for the community development and institutional support, probably the most exemplary and significant achievement was the formation of SHGs, predominantly of women members: there are now 2 071 SHGs - as against an original target of 920, with 33 056 women members, and 996 NaRMGs. Targeting of the Project and of uptake of individual activities has been reasonably effective, although benefits of some activities are inevitably enjoyed by all sectors of society. The means and application of targeting have proved justified and practical; but there is a concern that outreach to the poorest of the poor is in many cases limited. The gender balance of Project activities has been sound; and Project management, after a chequered start, has progressed significantly, and become highly proficient.  

The main implementation results under the agro-biodiversity and on-farm livelihood were: the reduction by nearly half of the area under jhum; development or improvement of terraced and irrigated lands; the strong uptake of diversified cropping focusing on horticultural and cash crops, as well as longer term plantations; integration of livestock into cropping and homestead production systems - for over 20 000 households; and community and watershed protection forestry. On biodiversity awareness achievement is disappointing.

In non-farm livelihoods, more than 1 600 enterprises, ranging from small shops to weaving factories have been set up, and commercialisation of small farming and cottage businesses is under way, although further work is needed to consolidate marketing, financing and business management aspects. Already over USD 235 000 is being borrowed from formal sources, with so far sound repayment history. This follows the success of SHG credit operations, whereby over 30 000 women members have savings totalling USD 238 000 and a track record of own funds lending, with 100% repayment.

In the social sector, construction and rehabilitation of water supply schemes has exceeded target by 50%; nearly 600 villages now have low cost latrine facilities; and the community health and education interventions, while diverging from those planned, have had reasonable impact.

The village infrastructure activities have exceeded the road building target by a factor of 9 and constructed 60 bridges or culverts and 25 roadside shelters, making a signal difference to access and transport facilitation. Some 82 community halls have also been built. In electric power supply, 80 out of the targeted 115 villages have been connected to the grid; those outstanding are now waiting for an imminent Government rural electrification programme that will greatly widen the coverage. Only one of the proposed 20 micro-hydro-electricity investments has as yet materialised.

In terms of Project performance, given the circumstances of the region and its peoples, the precarious nature of livelihoods and the prevalent state of serious under-development and progressive resource degradation, the relevance of objectives of the goal of the Project was - and remains - very high, a score of 6 on the scale of 1 to 6. The relevance of the other component objectives were also judged to be either high - 5, or very high - 6, with the exception of the non-farm livelihoods, which is marked only moderately high - 4. 

The effectiveness and efficiency of the Project can best be judged from its physical and financial progress. For the majority of activities, the Project has exceeded physical targets, sometimes by exceptional margins; principal shortcomings are in modification of jhum cultivation practices, health worker training and electricity supply, for which there are reasonable explanations.

In financial terms, current expenditure is only around 35% of available funds, with deficiencies for Village Infrastructure, non-farm Livelihoods and Natural Resource Management.  Expenditures have been well controlled overall; and reasonably balanced between districts, with an average of USD 1.55m per district, to a total of USD 9.29m.

The rural poverty impact is a function of the numbers of households and people actively involved and the extent of improvement in their livelihood and well-being. In the absence of explicit appraisal targets for many of the activities and of systematic M&E records of impact, best estimates of overall coverage, direct benefit and changes in wealth are summarised below:  

The Project has covered 862 villages comprising 39 203 households (HHs)  and 223 450 people, compared to the respective numbers of 460 villages, 23 000 HHs and 131 000 people planned at appraisal; and 1 000, 5 000 and 285 000 targeted (or implied) at MTR;

The numbers of direct beneficiaries by component or sub-component, based on data from five districts are: for the Institutional Support component – 33 165 HHs and 172 68 people; for livelihoods – 29 388 HHs and 152 041 people; in the Social Sector -  21 081 HHs and 111 119 people; for Village Infrastructure – 16 460 HHs and 82 887; and for Natural Resource Management – 22 073 HHs and 140 478 people.  With reference to these numbers and the evidence of field work assessments, the Evaluation Mission concludes that at least 25 000 households have benefited to date – and received substantive advantage from one or more Project activities; and

The wealth ranking aggregate figures are rough indicators of movement in poverty incidence, show that of a total of 18 390 households assessed between 1999 and 2004, the numbers of "poorest" have fallen from 9 742 to 6 455 and those "better-off" have increased from 172 to 625; thus, 18% of the relevant Project households has moved out of the poorest category; and 2% of households have moved up into the better-off category.  Village responses confirmed this fact, reporting substantive improvements in family economic and welfare predicament.

The main and most direct sources of poverty impact have been the production and income-oriented activities of on-farm livelihoods; crop diversification, irrigation and terracing; and to a lesser degree, non-farm livelihoods and income generation.  The key evidence of impact is that of assured food security and higher cash income for the great majority of participating households.  The benefits are manifest mainly in ownership of physical assets, resilience of family financial predicament and improved nutrition and health status, in particular for children and women.  Poverty impact by domain is scored as shown in the listing below:



Poverty Impact Areas     Ranking
       
Physical and Financial Assets     5 – high
Human Assets     6 – very high
Social Capital and Empowerment     6 – very high
Food Security     5 – high
Environment and Communal Resource Bask     2 – low
Institutions, Policies and Regulatory Framework     2 – low
Gender     6 – very high
Sustainability     3 – moderately low
Innovation and Scope for Replication     5 – high
Overall Impact     5 – high

The Project has exceeded, met or come close to meeting nearly all of the targets set at appraisal or modified at mid-term.  It has had a commendable coverage of smaller and poorer farm households and a significant impact on a representative cross section of the district population, of whom most subsisted below the poverty line pre-Project, and now many fewer do; and among whom the majority have been women. The only weaknesses in impact have been those related to environment, policies and sustainability.

The performance of the partners in implementation has varied. The performance of IFAD is seen as having been generally good, but the Fund could have been more supportive in rectifying some of the earlier deficiencies and obstacles to progress, IFAD is ranked as moderately high - The performance of UNOPS was not satisfactory in the first four years of the Project due to staffing and personal relations problems, but has since improved, although better consultation with the Project as to the expertise needed for specific supervision missions is required; a score of moderately low is given -

The performance of Government and line agencies has been constrained by an overly cumbersome structure and procedures and a certain disinterest or lack of collaboration among the local departments; a low performance - 2. Project Management has performed well, after a chequered start and is ranked as high - 5; and NGOs and CBOs have made a distinguished contribution to Project progress and impact and are given a score of very high - 6.

The overall assessment of Project performance is that NERCRMP has been a notably successful development intervention, despite the very difficult environment in which it was implemented and the hugely ambitious nature of its geographical and subject matter coverage. In particular, in the empowerment of beneficiaries and in the promotion and progressing of commercialisation of the small farm subsistence sector through the encouragement of savings, thrift and credit, farm enterprise diversification and private sector linkage, the Project has achieved exemplary results. On the principal IFAD measure of performance, rural poverty impact, it is clear that there has been a marked positive change in the predicament of both direct and indirect beneficiaries.

The most important insight that emerged from the present situation is that a huge burden of further work is still to completed, specifically related to bringing the 540 newer villages enlisted since 2003 up to the standard of the more mature villages. This leads to the recommendations:

  • that the Project restrict its activities to the current complement of 862 villages; and
  • that operating strategies and plans be made to ensure that all villages and groups reach optimal self-reliance and proficiency, including through the assistance of additional staff and/or external service providers.

Despite the success of the Project, there is an urgent need to consolidate the empowerment of women and groups by assisting them to formalise their rights to access to land; and to address the predicament of the poorest echelon of the tribal societies, who have not yet become fully involved. Accordingly, the recommendations are made:

    • that a study on the topic of land tenure be undertaken urgently and an action plan be produced whereby the Project will assist with advice on tenure registration and possibly with financial help in the form of a loan or matching grant aid for the completion of the necessary legal formalities; and
    • that a separate strategy be devised as a matter of urgency, for application across the Project districts and particularly among new villages to ensure the inclusion of the poorest groups.

In response to the need expressed by some village and group respondents for more resources for water supply provision, the recommendation is:

    • that the guidelines of expenditure per household and expenditure per component and sub-component be interpreted in a more flexible and pragmatic manner so as not to disqualify crucial infrastructure investments for smaller and poorer communities.

Key constraints to the effectiveness of Project Management in taking forward its demanding workload are its limited jurisdiction over district operations and the somewhat erratic and unreliable flow of funds. In this regard, the main recommendations are: 

    • that the RPSU is given more authority through the aegis of the DC to influence and control district activities in the latter years of the Project; and
    • that a new understanding and binding arrangements be thrashed out and agreed by all parties to remedy the deficiencies and delays in funds flow.

Notwithstanding the importance of the recommendations presented above, there is a series of critical issues facing Project Management and stakeholders, namely:

    • consolidating the achievements and gains that have been made in all facets of Project activity;
    • optimising the sustainability of gains, achievements and supporting development operations; and
    • facilitating dissemination and replication of the Project approach, modalities and interventions in other villages, blocks and districts.

To address these issues, the recommendations are:

    • that the Project make greater efforts in most districts to involve the block and district line departments in the development and empowerment of villages and groups. This could be done if necessary by invoking the more active support of DCs to familiarise the cadres with the Project philosophy and operations and provide on-job training and experience;
    • that a specific programme of liaison and knowledge sharing with block and district line agency staff be devised and rigorously pursued with a view to devolution of  Project tasks and responsibilities as the Project winds down; 
    • that a similar programme be mounted of active engagement with - and technical and financial assistance to - the NGO sector, directed at district level and possibly also regional level considerations, to also prepare them for potential handover of group fostering and service provision; and
    • that District Societies and the Regional Society be given greater responsibility for the planning, the raising of financial resources from Government, donor and NGO sources, facilitation and supervision of development along Project lines, by the allocation of Project funding and the deployment of some of the Project senior cadre, for up to two years after Project closure.  

Further and finally, taking account of the immense task of bringing the large number of new villages to an acceptable level of self-reliance - two elements are of prime importance in seeing through a satisfactory outcome to NERCRMP, not least in the 540 new villages. On this subject, the crucial recommendations are:

    • that as far as practicable, the current Project Management senior cadre be committed and contracted to continue in service up to Project closure, but not necessarily in the same positions; and
    • that the Project duration be extended for two additional years, as a minimum, to complete the operations and additional tasks now envisaged, that is up to 31 March 2008 for completion and 30 September 2008 for loan closure.

 


 1/ The Interim Evaluation Mission comprised: Jim Semple, Mission Leader and Rural Development Specialist; Somesh Kumar, Community and Institutional Development Specialist; Wayne Borden, Agro-biodiversity Specialist; Sarath Mananwatte, Economist: Livelihoods, Marketing and Credit; and Mini Bhattacharyya, Sociologist. The Mission was supplemented and its work greatly facilitated by inputs from the national counterpart team, comprising selected members of the Regional Programme Support Unit and the various District cadres. Flemming Nichols, the IFAD Lead Evaluator, joined the Mission for the final week of field work and for the wrap-up meetings.

 

 

LANGUAGES: English