Interim Evaluation: Agreement at Completion Point

Introduction

The IFAD Office of Evaluation (IOE) conducted an interim evaluation of the IFAD-funded Smallholder Cash and Export Crops Development Project (PDCRE) in Rwanda in 2010. In line with the IFAD Evaluation Policy, this interim evaluation was undertaken as a standard procedure in preparation for a possible follow-up phase of the project. The main objectives of the evaluation were: (i) to assess the performance and impact of the project; (ii) to generate a series of findings and recommendations that would guide a second phase of the project; and (iii) to provide an input for the Rwanda country programme evaluation (CPE) to be carried out in the fall of 2010. 

To assist in the evaluation process and to maximise the lessons from the evaluation, a Core Learning Partnership (CLP), consisting of IFAD and Government stakeholders, was established. Feedback gained from the CLP during the preparatory mission in May 2010 was incorporated into the Evaluation Approach Paper.  The main evaluation mission was conducted in June-July 2010. A final evaluation workshop was held on 16 December 2010 to discuss the evaluation findings and recommendations. This workshop informed and set out the understanding between IFAD and the Government of Rwanda for the Agreement at Completion Point (ACP).

The ACP is the agreement of IFAD Management and the Government to the main evaluation findings and their commitment to adopt and implement the evaluation recommendations within specified time frames. This agreement will be signed by IFAD management, the Director for East and Southern Africa (ESA) and the Government, the Permanent Secretary of the Ministry of Agriculture and Animal Resources. The agreed recommendations agreed upon will be tracked through the President’s Report on the Implementation Status of Evaluation Recommendations and Management Actions.  The role of IOE is to facilitate the ACP process leading to its conclusion.

Main evaluation findings

PDCRE comprised five components: (i) coffee diversification; (ii) tea development with two subcomponents (integrated smallholder tea development in the Nshili district and smallholder tea plantations in the Mushubi district); (iii) guaranteed credit to smallholder tea and coffee producers; (iv) development of new cash and export crops; and (v) project coordination and monitoring. The main findings of the interim evaluation are as follows:

Government strategy. Rwanda is an agricultural country, with 80 per cent of its 10 million population working in this sector. Unfortunately, the land area for the country’s 1.5 million farm families is very limited, and the average holding of farmland is only 0.5 ha. Consequently, Rwanda must use its land very efficiently, that is, find a strategy that allows it to use its comparative advantages, particularly the country’s tropical climate but still moderate temperature and its high altitude. One element of the Government’s strategy for fighting poverty and earning foreign exchange is to concentrate on promoting coffee, tea, and other selected export crops, the main subjects of PDCRE.

Political and economic changes. The project’s objectives and components remained identical to those set at appraisal, but many political and economic changes—some policy-related, others dictated by circumstance—have affected the project. One financier, BADEA (Banque Arabe pour le Développement Économique de l’Afrique), withdrew; the Government cancelled the contract with the only international service provider, TWIN Trading Ltd.; and the Government introduced a public-private partnership (PPP) policy, affecting the tea sector in the first instance. The Government also introduced a “zero-vehicle” policy for all development projects, established savings and credit cooperatives in all local administrative areas (Umurenges), and introduced a new cooperative law. After the mid-term, IFAD took over direct supervision of the project from UNOPS. These actions and the reformulation of the project at midterm made the project look quite different from the one appraised in 2002 and made more complex the evaluation mission’s assessments of the project’s effectiveness and efficiency.

Project relevance. The project was, and remains, relevant because its design was consistent with national policies and with IFAD’s strategies, and because it integrated lessons from other IFAD projects in Rwanda. The project addressed the needs of the rural poor and of women - both groups of special interest for the Government and IFAD. Although the project did not deal directly with food security, it addressed important dimensions of poverty such as income generation, human and social capital, and the empowerment and development of poor people’s institutions. The Government and IFAD understood the obvious needs of the rural poor and kept their strategies consistent, the project design simple, and the components appropriate, and they used the lessons learned from previous projects.

Project structure. It could be argued that PDCRE is a complex project because it has five components, three of which (coffee, tea and other export crops) are completely independent of each other, operate largely in different geographic areas and are implemented by different institutions. However, the evaluation mission considered the project relatively simple, especially once its coordination was centralized (as enforced by the Government’s zero-vehicle policy). At that time a separate coordinator was assigned for each component, and because nearly all project operations were outsourced to specialized agencies, the situation was, in effect, similar to having three separate projects under the same program manager and monitoring and evaluation unit, supported by one financial intermediary (Development Bank of Rwanda) under the credit component.

Overall results. Overall, PDCRE has been a moderately successful project (it continues until September 2011), with the coffee components achieving satisfactory results earlier than the other operational components have done. In addition, the tea component shows promise of achieving substantial results before the project ends, and the project has had a catalytic effect on sericulture production, helping MINAGRI to establish training centres and financially supporting four sericulture cooperatives. However,  performance under the project has been uneven in many respects, with some coffee cooperatives and their processing facilities becoming profitable after only a few years of operations, while the operations of the other coffee cooperatives, tea complexes and sericulture cooperatives must expand and become more professionally run before they can be proved to be profitable. It is also necessary to improve productivity and quality of coffee and tea. This can be done through extension, increased use of farm inputs, and incentive-level prices to farmers.

Agreement at completion point

The interim evaluation made a number of recommendations for a potential follow-on project, which IFAD and government are in the process of designing. In finalizing the design, the recommendations by the interim evaluation will be taken into account.

Recommendation 1

• The next phase should emphasize expanding the capacity to process coffee.
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be implemented during the final design of the Project for Rural Income through Exports (PRICE).

Recommendation 2

• The next phase should expand the training capacities of central cooperatives agencies.
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be implemented during the final design of PRICE.

Recommendation 3

• The next phase should support SACCOs as a means to facilitate financial services for small farmers.
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be implemented during the final design of PRICE.

Recommendation 4

• The development goal of the next phase should include an institutional element as institutional sustainability is required.
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be implemented during the final design of PRICE.

Recommendation 5

• In the next phase strengthening cooperatives and their management should be a priority, starting with a detailed analysis of management’s shortcomings and of gaps in the skills and competencies among the members, committee members and staffs.
  
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be addressed during the final design of PRICE. However, ESA and Government disagree with the notion that the management shortfalls of cooperatives need to be studied in detail before designing an institutional strengthening programme. In fact, the shortcomings are well known: firstly, transparency of management and governance structures, and secondly, administrative and business management deficiencies. To address these, the Turnaround Programme has developed a thorough approach and sound tools, culminating in a custom-made support programme designed to build up the management capacities of each participating cooperatives over a period of 6 to 18 months.

Recommendation 6

• The evaluation mission recommends a number of other studies: (i) a study on the environmental effects of the coffee-washing stations and tea factories; (ii) an updated study on “other cash crops” because the earlier one is 6 to 7 years old and the economy of sericulture appears uncertain; and (iii) a cost-benefit analysis of organic coffee production in Rwanda coffee cooperatives.
  
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be implemented during the implementation of PRICE. However, ESA and Government disagree with the selection of proposed studies: (i) the environmental effects of the coffee washing stations and tea factories are known (effluent water at CWS and wood fuel use at tea factories) and PRICE will address these by installing water recycling systems at 45 existing CWS and their inclusion at all new ones. OCIR-Café has the relevant experience. The wood use at tea factories will be governed by national laws that are sufficiently tight to ensure sustainable regrowth of timber; (ii) an updated general study on “other cash crops” would be of little practical relevance as the related investment decisions need to be guided by private business interest; and (iii) a cost-benefit analysis of organic coffee will only make sense once certified organic coffee will have been sold for some time, thus this study has not been specifically included in PRICE.
  
  Instead, the agreement is as follows:
  

  To add to the environmental benefits of the CWS, IFAD and government agreed that a coffee pulp transformation study be carried out to support the production of compost from coffee pulp at the CWS;

Instead of an updated general study on “other cash crops”, the project would include market analysis and value chain analysis for promising value chains in the horticulture component; while an Intermediate Sericulture Assessment would be carried out after 3 years of PRICE support to the subsector; and

Two detailed studies to prepare the farmer field schools (FFS) methodologies for both tea and coffee will be commissioned instead as part of the preparatory activities for PRICE. In addition, unspecified technical assistance allocations are provided for studies that may be required during project implementation.

Recommendation 7

• The next phase of the project should work to ensure that the PPP is pro-poor.
  
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be implemented during the final design of PRICE, mainly in the form of innovative tendering procedures for tea Greenfield sites, culminating in price agreements between factory owners and tea cooperatives, based on a percentage of the made tea price.

Recommendation 8

• In the next phase the project should introduce a “money-for-work program”.
  
  Deadline date for implementation: 1 July 2011
  Entities responsible for implementation: IFAD
  This recommendation will be implemented during the final design of PRICE, mainly by including an alternative income scheme for poor farmer households participating in the coffee expansion and the tea planting programmes.