Mid-term evaluation

Rwanda has one of the highest population densities in the world (280/km2 ) and a very high annual population growth rate (3.3%), the result is that land access is becoming increasingly more difficult, and farms are shrinking in size all the time. During 1965-1989 agricultural production kept pace with population growth, mainly by increasing cropped acreages, while yields stagnated, but this trend was reversed in 1990, when the country had a food deficit, aggravated by civil war. The project zone is in the Gikongoro Prefecture and comprises 7 of the 13 municipalities in that Prefecture; Gikongoro, in the southwestern part of Rwanda, has a population of approximately 500,000 and is located mostly at elevations above 1,800 m. The Prefecture has quite a favourable rainfall pattern (1,600 mm per year divided into two rainy seasons), but some of the soils are acidic and the steep topography can lead to problems of erosion. At the higher elevations there are areas of severely degraded pasture, which is virtually sterile, whilst in the valley bottoms there are about 2,000 ha of swamps which require extensive reclamation before they can be farmed and are also of very low fertility. Food crops grown are plantain, sweet potatoes, cassava, sorghum, and maize - coffee is an important cash crop. Livestock is widely dispersed but makes only a small contribution to farm incomes.

Project design and objectives

Since 1976, UNDP has financed a series of projects in the Prefecture, which were concerned with reforestation and soil conservation, farmer training, and agricultural intensification, particularly for wheat and potatoes. The Gikongoro Agricultural Development Project (GADP) builds on the last in this series of projects: the Project for the Intensification of Agriculture (PIA).

Target group

Gikongoro is the poorest Prefecture in Rwanda; per capita farm incomes in 1988 were estimated at USD 85, which is less than one third of the national average. The target group of 42,000 farm families were classified as (i) farmers cultivating less than 0.5 ha, which accounted for approximately one quarter of the target group, (ii) farmers cultivating between 0.5-1.0 ha, which accounted for over 40 % of the target group, and (iii) farmers cultivating between 1.0-1.5 ha. Because of the very small holdings it was important to consider diversified on and off-farm activities to try and improve the incomes of the poorest group, which was also to be given priority in the allocation of new land developed. It was estimated that in about 60% of the target group, women were either heads of the household or the main farm operators.

Objectives and components

The objectives of GADP were to improve food security and raise small farmers' incomes on the one hand, and to strengthen the agricultural services in the Prefecture on the other. To achieve this, the project intended to introduce agricultural intensification, (food crops and above all cash crops such as wheat and potato), raise livestock production, develop soil conservation methods, and improve soil fertility. It also set out to strengthen the small farmers' groups and provide them with access to agricultural credit. Project components were (i) research and development, including monitoring and evaluation, (ii) vocational training, (iii) agricultural extension and support activities (seed production, agro-forestry plants and livestock production), (iv) agricultural credit, (v) upland development, (vi) valley bottom development, (vii) track rehabilitation and construction, (viii) construction of stores, and (ix) the promotion of micro-enterprises.

Total project base costs were estimated to be USD 24.2 mn, plus an additional amount of USD 7.0 for contingencies; these were allocated as follows:

Extension/training	USD 5.2 (22%)
R and D, M&E	USD 1.7 (7%)
Institutional Strengthening/Project Management	USD 2.3 (9%)
Group Support/off-farm enterprises	USD 3.6 (15%)
Land Reclamation	USD 2.5 (10%)
Rural Infrastructure	USD 2.0 (9%)
Marketing and Credit	USD 6.9 (28%)

The project was to be financed by IFAD (36%), UNDP (10%), UNCDF (12%), the beneficiaries (25%) and the Government of Rwanda (17%). The UNDP element was to finance the TA, cooperative development and training activities, whilst UNCDF funded part of the civil works, storage structures, land reclamation etc. IFAD funds were to be used for development of the extension services, some storage and land reclamation costs, research and development and M&E.

To promote the marketing of products and inputs by means of small farmer groupings, the project contracted CFRC-IWACU (Centre de Formation et de Recherche Cooperatives IWACU) to train groups to become autonomous farmer cooperatives. The project's credit component was to be implemented through the small scale Banques Populaires (BPs), subsidiaries of the national Cooperative bank; to do this required extension of their coverage in the project area and also strengthening of the services offered. In fact the project defrayed 50 % of the BPs costs of operations for the first three years, and also provided an emergency fund to cover bad debts in this period. The detailed design of the credit provision was undertaken during implementation, in order to obtain a specific focus on the needs of the poorest members of the local communities. The project provided funds for the adaptation of technologies for food processing, wood crafts and blacksmithing that could create employment opportunities, and also offered training to entrepreneurs and traders for local enterprises such as transportation, storage, goods supply etc. Small ruminants were given special emphasis, in terms of the introduction of upgraded stock, as a means of providing income earning activities.

An adaptive research programme was devised which was intended to fully involve the farmers in selecting the activities to be undertaken. The programme was to be carried out by the Agricultural Research Institute of Rwanda (ISAR), and would be monitored by a joint Adaptive Research and Monitoring and Evaluation Unit. This programme would pay particular attention to the problems of soil erosion, conservation and soil fertility. In addition the programme would extend to examining land use issues such as tenure patterns, reclamation and costs and returns from conservation activities. A trials and demonstration programme on farmers fields would also be established, linked to the extension services. To ensure that high quality seeds were available to farmers, the project would promote the establishment of privately run nurseries, especially for wheat and potatoes.

The project supported a new extension system at Prefecture level for the seven communes in the project area, based on a modified T & V system using groups. Women's concerns were given high priority in this system, and two-thirds of the extension agents were women. Under this heading the main financial outlays were for training, transport and field allowances. Land reclamation of the uplands or poorly drained lowland swamps was beyond the capacity of individuals or groups and required well designed and extensive schemes; the project proposed to finance a selection of such schemes in order to increase the available productive land. These developments would also involve large inputs of beneficiary labour in the construction of drains and other earthworks. The rural infrastructure programme would finance the construction of access roads to communes which had no vehicular access, plus rehabilitate other selected roads to aid access and marketing.

Expected effects and assumptions

The project was expected to result in increases in smallholder incomes of more than 50%, and improvements in the family nutritional status. In addition surplus crops, meat and milk would be made available for consumption locally. The very poorest would also be eligible to receive parcels of reclaimed land as it became available; this was especially expected to be of benefit to women. Unquantified benefits included improved conservation (hence increased environmental protection) and employment generation, better extension and stronger farmers organisations. Assumptions made in the project design included (i) the comparative advantage of the Gikongoro region for wheat and potato production, (ii) the ability of the farmers to undertake their own marketing in a relatively short time, given basic guidance, (iii) the assumption that the adaptive research activities could rapidly provide extension messages for the new extension service, and (iv) that the Banques Populaires were prepared to become involved in the provision of a large number of small loans, and provide advisory services to their clients.

Evaluation

The MTE mission met with representatives from the Ministry of Agriculture, Livestock and Forests (MINAGRI- the implementing agency), project staff, the local authorities, and other agencies implementing different aspects of the project, farmers and others helped by the project. Various visits were made to the project area and initial findings were discussed with the Minister of Agriculture in Kigali.

Implementation context

From the beginning the project benefited from the inheritance of the UNDP projects which had immediately preceded it, especially PIA, which was concerned with the intensification of agricultural production, and from which GADP inherited vehicles, equipment and a functioning organisation. The TA has been of a high standard and has made a positive contribution to project implementation. The most negative factor, from October 1990, has been the war in the northern part of the project area; this period was marked by difficulties in communication and travel. In addition promotion of the production systems have been affected by the three changes in MINAGRI which took place, which made long term planning impossible. In the Gikongoro region the introduction of seed potato infected with bacteriosis in 1991-92 resulted in a severe loss of confidence by the farmers.

Project achievements

Several positive results must be credited to the project, and in particular: (i) the strengthening of the agricultural services in the Gikongoro Prefecture; (ii) the sound incorporation of the project into the new administrative structures of MINAGRI at prefecture level; (iii) the assistance to set up effective training entities in the Prefecture, such as IWACU and INADES/; (iv) the sensitisation of the farmers through the group training schemes carried out by IWACU, which holds out great hopes for sustainable development; (v) the large number of farmers who have been trained to manage the cooperative groups and the numerous project agents trained; (vi) the improved familiarity with the project zone, through surveys and enquiries carried out by the M&E Unit team; (vii) the research on the impoverishment of the farmers in the project zone and the means of curbing or preventing it; (viii) the dissemination of the runner bean, which is more productive and resilient than the dwarf bean, and (ix) the establishment of an input distribution and storage facilities network, even though these functions were still dependent on the presence of the project.

The MTE found that the GADP was too complex a project to be implemented with facilities and resources planned for it, with multiple components and important parts to be sub-contracted that require guidance and direction (and for which the necessary skills -in the case of the roads component- either do not exist, or are of limited local interest -the case of the micro-enterprises component). One major problem has been the lack of coordination between the various services and with the sub-contractors. The failure to appoint the Deputy Project Manager and set up the Promotional Office (which was responsible in the project design for coordination) contributed to this state of affairs. The financial management of the project is less chaotic than it was during the first three years thanks to the new accountant who has made it possible to carry out an external audit, but the performance is still far from satisfactory.

Production of food crops has varied enormously, for example soya has achieved 95% of the target set at appraisal, whilst sorghum has achieved 526%. The acreages under sweet potatoes, a food security crop par excellence, have almost doubled since 1989 substantially improving nutrition and the cultivation of runner beans, which was not even mentioned as a possibility in the appraisal report, has given spectacular results.

The M&E unit has only partly played its role, spending too much energy monitoring the project's management indicators. In particular, it has failed to help the R&D team to identify the small farmers' main constraints, monitor the impact of the responses and draw up a plan of action. Little attention has been given to targeting the actions undertaken. The small farms and the women have remained unaffected.

The R&D Unit has not played the role assigned to it, which is to understand the way the farms operate, define the development strategy and to provide extension input on relevant themes. Techniques are rarely monitored and checked, and they have not been adequately translated into themes for extension. Field work has mainly consisted of tests and trials in a controlled environment and only secondarily in the rural environment. Few of the proposed innovations are rational alternatives for the farmers because of the constraints under which they labour. The intervention envisaged in the project design was based on a farming systems approach, but this has not been put in place. The constraints on the farms have not been analysed, or prioritised and no solutions have been proposed.

The GADP has used high calibre subcontractors and useful results have been forthcoming, particularly with regard to vocational training and supporting the small farmers' groups. Since mid-1992 the vocational training service has been linked to the R&D-training division, probably to facilitate the dissemination of technical themes. However, training has different objectives, which are hampered by this institutional framework. The project has not been given the facilities needed to guide or direct these sub-contractors, and to ensure that their strategy is best aligned to the project strategy.

The production and marketing of wheat, potato and runner bean seed by individual farmers and groups has declined overall because these activities are not profitable, potato seed has been contaminated by bacteriosis, and the markets were found to be too small. However, many small farmers said that the inadequacy of high quality seed was a major problem. GADP has also failed in its attempt to privatise the nurseries, apparently because the potential markets were far smaller than forecast, and the prices proposed by the project were not remunerative.

The mission noted that the Banques Populaires (BPs) have scant interest in socio-economic development in the rural areas, with the result that the volume of loans has been very small. As far as the IFAD loan is concerned, GADP has already paid the BPs about 20 times more under the guarantee scheme, than the losses recorded by BP due to delinquencies. In addition, the investments likely to benefit from loans are at risk (the difficulty of selling the wheat produced, and the bacteriosis of the potatoes), the conditions for invoking the guarantee funds are not always described and many individuals and groups are no longer eligible because they have failed to repay their debts contracted with the aid of previous projects. Very little effort has been made with regard to awareness- building and there has been little contact between the BPs and GADP. Loans only cover inputs, whereas the small farmers have other priority needs (particularly small livestock.

The plan to reclaim upper pasture land to develop 380 ha in 7 years was fully justified considering the shortage of land, but it has been a failure (wastage of inputs, embezzlement of subsidies, risks regarding soil conservation). The land was abandoned as soon as the project stopped providing the inputs, because the farmers could not afford lime and fertilisers, and without these inputs the land was non-productive. In 1992 the programme was halted and replaced by a root-terracing construction programme over 60 ha. The mission found that the techniques had not been fully mastered, the development had negative repercussions on soil conservation, and the scheme was targeted at the wealthy. The reclamation of areas at the bottom of the valleys for development have been reduced from 450 ha to 160 ha, because the project encountered a range of different problems, the most important being tenurial (redistributing the lands to the benefit of the poorest people) and organisational (the beneficiaries were reluctant to take part in maintaining the facilities).

The roads component has been considerably delayed mainly because of late deliveries of materials, and only 12% of the component has been completed. There have also been budgetary over-runs. It is expected that the roads would be completed by the end of 1995 (at the rate of 40 km/year). Maintenance is a major problem for the roads and tracks; a protocol with the municipalities that will gradually take over maintenance from GADP only comes into force once a first section has been delivered, meanwhile the tracks are rapidly deteriorating. The storage facilities construction programme (26 stores) was scheduled in expectation of increased production of cash-crops (potatoes and wheat), but as this has not happened the programme now seems to be over-sized. The problem is to find uses and users for these stores in order to ensure a return on the investment. The micro-enterprises component is working satisfactorily, in that the micro-enterprises being supported are profitable.

Effects assessment and sustainability

GADP has had a limited impact on agricultural development, and even less than its predecessor, PIA. Input consumption is still very low and few technical themes have been proposed that are within the reach of the small farmers. Production of the two cash crops supported by the project has not risen as forecast in the appraisal report: potato production has been seriously affected by bacteriosis, and failure to remedy it has led to a slump in yields, while wheat output is stagnating well below its potential levels and is likely to be affected by the withdrawal of subsidies by the project, which until now have covered the transport costs of inputs and products. Although beans and sweet potatoes should give an improvement in the nutritional value of the family diet, and production of both these items has increased, the extent to which this has reached the most vulnerable very small farmers was extremely limited, because of failings in the extension activities. It is probable therefore that neither food security nor nutritional levels have yet improved significantly, and this should be one of the major aims in the time remaining to the project.

Given the limited period of implementation, the MTE did not consider that there would have been any effects on income levels. The impact on women did not appear significant - other than the stablishment of a quota of 30% recruitment for extension workers. Overall, the project actions were estimated to have been neither positive nor negative for the environment, which was surprising given the fragile nature of the environment and the pressure to incorporate some conservation activities in the project.

The mission considers that the R&D Unit cannot achieve the set objectives before project termination, because it has only now begun to diagnose the constraints and suggest solutions for the production systems under the responsibility of the M&D unit.

So far from the micro-enterprises component have failed to offer any solutions to the problem of under-employment, because only a few cases are being financed (38) and few jobs are being created by each enterprise (2.5). In addition, the project has found that the identification and training costs are high and that success is dependent on high quality inputs at all stages (i.e. proposal, appraisal, monitoring, etc).  One indispensable condition for success appears to be regular monitoring, support and training but the beneficiaries are generally people who are already more privileged than most people in the local area.

Extension work has had a very small impact measured in terms of the number of farmers involved and the rate at which the extended techniques are being adopted, and this has contributed to the stagnation or decline in cash crop production. The apparent causes of this weak impact are: (i) few new technical themes have been provided by the R&D Unit; (ii) the field agents disseminate these themes uniformly, without taking account of the farm features need specific responses; and (iii) the themes being disseminated do not match the small farmers' priorities and are outside their reach. Small farms and farms headed by women have practically been bypassed in terms of extension work. For the small farms, the R&D Unit has been unable to advance any proposals for maintaining fertility using methods which need costly inputs which the farmers cannot afford. The mission noted a reduction in the use of erosion control techniques, probably because the local supervisors have failed to motivate the small farmers to continue applying the techniques that were formerly mandatory (e.g. erosion prevention ditches).

Main issues and recommendations

The following major recommendation were made by the MTE mission:

• IFAD should do everything possible to ensure the financing of the technical assistance and vocational training components which the mission believes to be indispensable, until the end of the project.
• So far, coordination between the various services and between the sub-contractors has been very inadequate. It is essential for the project to be reoriented to guarantee maximum synergies between the services and the components.
• The project discovered the need for a participatory approach somewhat late in the day, and then through the intervention of an NGO (IWACU, to support the groups), or through technical assistance and the United Nations volunteers in the case of rural tracks and valley bottom reclamation. It is important for this form of cooperation to be given financial support until the end of the project to reinforce the message of participation.
• The following must be done rapidly for the administration and financial monitoring of the project: (i) the accounting system must be fully computerised; (ii) the accounting service personnel must be trained by the technical assistant acting as chief accountant; (iii) the accounts for salary advances and loans for the purchase of motorcycles must be audited; (iv) the stocks must be restored and management procedures revised; and (v) punishments must be meted out for embezzlement.
• Priority must be given to completing the survey of farm production systems. While awaiting this, R&D must be partly modified by redirecting efforts to focus on the major constraints encountered by the small farmers.
• The mission deems it necessary to review the R&D programme and recommends the following:

• the R&D unit should cooperate with the M&E Unit to programme activities;
• a participatory approach should be adopted to select priorities and to design extension messages;
• the target groups should be redefined and technical proposals made which meet the needs of small farms less than 0.50 ha in total size, and of the women;
• the most promising innovations must be selected for these very small farms by giving priority to increasing organic matter and combating erosion;
• the innovations proposed must enable the small farmers to gradually change their production systems;

• The project should give priority to enabling small livestock farmers, who have been left to one side so far, to keep or increase their herds. The small livestock farmers should be organised and grouped together in order to obtain mass preventive treatment, such as vaccinations, from the veterinary officers.
• The mission recommends that IFAD suspend all further activities and all commitments with the BPs, and send a mission to Rwanda as soon as possible to review and redefine the agricultural credit programme.
• The MTE mission agrees fully with the cooperating institution that the upland pasture component, which has proven a failure, should be terminated. A report should be filed so that an in-depth evaluation can be carried out at the end of the project and the results obtained should be exploited and the causes of failure identified.
• The valley bottom development programme should (i) keep the water management expert responsible for this component until the end of the project; (ii) pursue and broaden cooperation with INADES for the organisation and the training of the assignees of the marshlands and strengthen cooperation between the teams of women leaders and technicians; (iii) make provision for IWACU to support the marshlands committees and groups of farmers in order to consolidate their organisation and operations, and to maintain the water management facilities.
• The mission agrees with the cooperating institution and the mid-term review mission that the subsidies on wheat should stop in 1994. The subsidies were justified as a transitional measure while waiting for infrastructure scheduled for the short term. At the present time these subsidies can no longer be justified because the infrastructure has not been implemented.

Lessons learned

GADP has turned out to be too complex a project, making it very difficult to implement and coordinate, due to the number of components, the number of sub-contractors involved, and the number of financiers. Several components have no direct relationship to agricultural development in the Prefecture and could have been designed separately (for example micro-enterprises). IFAD should avoid designing such complex projects and should not add any components that have no significant or direct impact on the project objectives.

The parties involved in GADP tend to consider sub-contracted work to be of no direct concern to them, and the project has failed to take any measures to closely monitor the sub-contractors. Sub-contracting is a sound way of limiting the activities directly being implemented by the project, but it also requires very close monitoring in order to ensure that the activities fit in with the project strategy.

From the project design phase onwards it is essential to make provisions for the maintenance of roads as soon as they are completed. New or rehabilitated tracks rapidly deteriorate if the maintenance is not carried out from the moment they come into service.

The main lessons from the micro-enterprise activities were: (i) it proved costly to identify, train and monitor the promoters, as this required a large team of well-trained personnel; (ii) the beneficiaries were generally people who are already more privileged than the IFAD target group, and (iii) the impact on employment was limited and possibly only significant in the long term.

The BPs are very sound institutions for collecting rural savings but they were not interested in contributing to (and playing an active part in) rural development. The BPs seemed appropriate vehicles for credit to the project designers, but weren't, and this hindered project benefits going to the poorest. In future such an assumption should be tested in a pilot phase.

The work being carried out by IWACU in vocational training is very important and is beginning to show results. Since the work is a long-term development process, it was unrealistic to expect quick results and to design an activity with such a limited timeframe. Little attention has been given to targeting the actions undertaken. The small farms and the women have remained unaffected. Vocational training must be planned as a long term activity.