Completion Evaluation

Given that a second phase of the Small Farmers Credit Project (SFCP) for Jamaica (Loan No. 100-JA) was under consideration, IFAD's Monitoring and Evaluation Division carried out a completion evaluation of that project. The field work was combined with an Identification Mission organized by the Fund's Latin America and the Caribbean Division, in order to ensure that the results of the Evaluation Mission would be effectively incorporated into the design of a new project. In fact, a main recommendation of the Evaluation Mission was that there should not be a second phase of the SFCP. What is recommended, instead, is a project in which the emphasis should be on the diversification of financial services for the rural poor.

Background

In the early 1980's, Jamaica's economy was in a critical situation due to the long-term decline in export prices for its bauxite-aluminia mining and processing industries. The need to reactivate traditional agricultural production to boost exports and reduce imports, especially of food products, motivated the Government of Jamaica (GOJ), to assign a more important role to agriculture in its policy decisions. It was assumed that if sufficient credit and technical assistance was provided to the country's small farmers (those with farms under 25 acres), they would then be in a position to apply improved production techniques, thereby increasing yields and production, and achieving the stated policy goals.

The situation of the agricultural credit institutions at the time, however, was very precarious due to a longstanding policy of providing funds to farmers at subsidized interest rates and accepting low loan recovery rates. The institutional organization consisted of a national level agricultural bank (the Jamaica Development Bank (JDB) up to 1982, and the Agricultural Credit Bank (ACB) thereafter), and over one hundred Peoples Cooperative Banks (PCBs) at the local level.

The GOJ invited IFAD to assist in developing a rural development project for the country focusing on agricultural credit. The Fund requested the Inter-american Development Bank (IDB) to carry out the appraisal and to cofinance the project. The final design followed previous IDB experience with credit in Jamaica. During appraisal, the need to change the existing institutional structure led to some delay in completing this phase. IFAD's Executive Board approved the project in late 1982 and IDB's Board did likewise early in 1983. Project start-up took place in mid-1983, and the final closing date was December 1989.

Project design characteristics

The project had dual objectives. First, to strengthen the agricultural credit delivery system, by providing the first important support to the new ACB. This was done by supplying loanable funds with an interest spread which was assumed would be sufficient to cover the costs to ACB of administering the sub-loans. This spread was deemed also sufficient to cover the costs of the PCB's in their role as financial intermediaries. Secondly, the project strategy proposed, through supervised credit and the introduction of improved farming techniques, a substantial increase in crop yields of small farmers, thus generating higher incomes from on-farm activities.

The project design proposed a supervised credit system, with ACB-supported farmers receiving technical assistance from the Ministry of Agriculture (MINAG), both in the introduction of improved farming techniques and in soil conservation works, to allow for an increase in short- and long-term yields. Project beneficiaries would consist both of institutions (ACB, PCBs and MINAG), as well as small farmers cultivating between 2 and 10 acres of land. Institutional coordination was a responsibility of ACB as executing institution, and resources were assigned for project monitoring and evaluation (M&E) activities, to be carried out by MINAG.

The total cost was estimated at USD 25 million, with GOJ providing 20% and IDB and IFAD each 40%. The project would be implemented over a four year period, concentrating activities in four distinct geographic areas corresponding to four watersheds.

Once under implementation, the project underwent considerable modifications from its original design. In particular, two major changes were introduced. During the first year, GOJ requested a change in project focus, from the limited pilot-project area originally designed, to a nationwide, all-island encompassing project. A further change, introduced also at GOJ's request in the second year, was essentially aimed at widening the target group, by increasing the upper size limit of eligible farmers from 10 acres to 25 acres. GOJ's motivation in asking for these changes, reflected the new policies of promoting export and import substituting crops. It was also justified as necessary given the low level of project disbursements in the early years and furthermore, the large devaluation of the Jamaican Dollar (JMD) that took place in November 1983, had increased the amount of loanable funds threefold in the short run. These changes to the project's internal design logic were very important, and consequently also complicated the evaluation of project results.

Main project results

The project was actually implemented over a six year period, 50% longer than originally planned. IFAD's loan was 87% disbursed when the project closed in December 1989. The main results will be presented below, both in terms of effects on institutions as well as on the small farmer beneficiaries. The role of loan supervision on project implementation will also be discussed.

Effects on ACB. With the project, ACB has been developed and strengthened as a viable and profitable credit institution. However, it must be pointed out that ACB's main source of income has increasingly been investments in government securities and fixed deposit accounts, rather than income from its agricultural credit operations. Without the income from these non-agricultural credit activities, the ACB would not be the financially successful institution it is today. In addition, ACB has benefitted from not having to absorb the foreign exchange risk. Secondly, the ACB has transferred the risk of sub-loan default mainly to the PCBs.

Effects on the PCBs. Under the project the loan portfolio of all 42 participating PCBs expanded very substantially, but these banks on the whole did not benefit. A limited number of PCBs (about 10 out of 42) did improve their financial situation in relation to what it had been previous to the project, but in 75% of the cases their losses increased. The PCBs in the aggregate remain weak financial intermediaries, many now facing high arrears rates.

Effects on MINAG. During some years, especially when its field staff was being reduced due to cuts in its budget, the MINAG benefitted from the project, as it received funds for staff, operational expenses and for the purchase of vehicles. MINAG was thus able to maintain its services to farmers, although these drastically fell off once project funds were exhausted. In terms of improving MINAG's capability to introduce changes in farm technology, the project did not have much success. Yield increases were meager and on-farm soil conservation adoption was disappointing. Community soil conservation works carried out did exceed the original project goals, but unfortunately these were not integrated into other project activities, so their overall contribution to attaining objectives was of little relevance.

Effects on Project Management, Monitoring and Evaluation. While the managing of the individual components was in general adequate, overall project management and coordination was weak. The monitoring and evaluation activities carried out served mainly to record events that took place but were of little use in assisting management.

Effects on Credit to Small Farmers. The project contributed very substantial funds for small farmer credit, supplying about 30% of all ACB funds in the years 1984 to 1987. Thereafter, project funds constituted a small portion of the total as other external sources of funds were supplied to ACB.

In terms of the number of sub-loans, some 14 150 of these were granted to small producers. However, since ACB was not able to provide the number of actual beneficiaries (many repeat borrowers occurred), the Evaluation Mission arrived at an estimate that between a minimum of 8 500 to a maximum of 9 800 farmers were effectively reached. This compares favourably with the 4[500 that were targeted at appraisal.

In 92% of cases the sub-loans went to farmers that possessed a total area of under 25 acres (the definition of small farmer used by ACB), and all sub-loans complied with the Credit Regulation which indicated that the area cropped be under 25 acres. With regard to the original target group (those farmers having under 10 acres), 77% of all sub-loans went to this group. Available data suggests that 51% of sub-loan recipients increased their incomes and 64% increased their demand for labour. The Evaluation Mission's findings tend to indicate that sub-loan beneficiaries were farmers who, while poor, had better access to land and other resources.

Project credit funds went primarily to the financing of export crops (50%), to the purchase of livestock (30%), to food crops (16%), and for farm infrastructure. This distribution corresponds to GOJ's stated priorities. There is no evidence, however, that the project contributed to increasing crop yields as originally planned at appraisal. While production was increased through the use of credit, this was achieved by increasing the area cropped.

Project Supervision. In general, while field supervision by IDB was adequate, with multiple visits being made to PCBs to ascertain that project activities conformed to those included in the loan agreements, there was insufficient supervision of other loan related matters, such as the use of loan recoveries for providing further credit to farmers. The major implications of the changes suggested by GOJ and approved by IDB were not adequately anticipated. The extent to which those modifications could alter the project's original focus was not anticipated.

Conclusions and lessons learnt

The overall assessment of the project was complicated because of the important changes introduced during implementation. These changes can be seen as an adaptation of the project to the GOJ evolving priorities. But, at the same time, because of these changes, the project as implemented did not reach its initial objectives. Nevertheless, given the achievements described in the report, both the GOJ and the IDB have considered the project successful. In addition, the project has yielded a number of important lessons for future projects, the most important of which are presented in the following paragraphs.

Lessons Concerning the Macro-economic Policy Environment. In the case of Loan No. 100-JA, there is a clear lesson for IFAD on the need to monitor more closely the evolving economic situation in the countries, in order to adapt its projects when important policy changes take place. When this type of situation occurs, the Fund should be flexible enough to redefine projects when the original design has become obsolete, but taking care that the needs of its target group are not adversely affected.

Lessons Concerning Definition of Beneficiaries. IFAD should have a clear understanding of country priorities with regard to the target group before carrying out the preparation and appraisal of a project. This requires that at identification, the definition of which groups will be the beneficiaries, should be worked out in detail with the staff of national institutions that will later implement the project. Eligibility criteria to be applied to project beneficiaries should be operationally applicable and as precise as possible.

Lessons on Project-proposed Technical Solutions. The credit needs and production possibilities of the target group have to be analyzed critically in greater depth, to ascertain if the project's proposals correspond to the beneficiaries' actual needs. The increased production capacity of poor small farmers should be based on a realistic assessment of available improved technology for their type of land, their crops and their resources. The project results indicate that a mistake was made in proposing exaggerated increases in production and yields, when the technology available was only adequate for producing modest increases. The capacity of the extension system to disseminate new technology, should also be carefully assessed. Provision should be included in future projects to periodically review the proposed project's crop and technical packages, to ensure that agronomic recommendations remain in harmony with changes in input and output prices.

Lessons on the Need for Diversified Financial Services. The project's focus on credit delivery mechanisms alone should not be repeated, as no attention was paid by ACB towards generating savings. A balanced approach is required to promote savings in rural areas, thus reducing the need for additional external funding, while providing new loans in a manner such that higher loan recovery is achieved.

Lessons on Building up Institutional Capabilities. The capacity of institutions to adapt to new project demands can easily be overestimated. The development of institutional capacity requires time and resources which the project should provide for. Institutional support services to the project, from agencies not used to providing these to development projects oriented to rural poor populations, should not be left as the undetermined responsibility of the government, as it is easy for the latter not to take the actions required. The insufficient support to the PCBs is a case in point, as there was no explicit provision in the project concerning how these were to be strengthened and who was responsible for this.

Lessons on Project Monitoring. Project monitoring should be explicitly integrated with project management. Otherwise the result is a divorce between the monitoring staff and the project manager who is the direct user of monitoring information, and no positive benefits are obtained.

Lessons on Project Evaluation. The limited usefulness of the evaluation studies based on farmer surveys, highlights the need to make greater use of loan application forms and other project records, in carrying out evaluation activities. Surveys should be used to complement these other sources of data, and care should be taken to ensure the use of short questionnaires that refer only to that data not available from other sources. The length of time required to process survey data would be reduced and non-sample errors would be minimized. Case studies should also be used.

Lessons on Project Sustainability. From the previous lessons, it is clear that the project has a reduced long-term capacity to continue making loans to small farmers. Given the high rate of arrears, which evolved from an average 62% in the first two years of the project's implementation to an average of 28% in the project's last two years, the loan funds will continue to diminish.

Nevertheless, the project did allow ACB to strengthen its financial situation, by using sub-loan recovery to purchase government securities and invest in fixed term deposits. This use of the funds, while unorthodox, did permit ACB to generate income to continue to function, and thus would allow the further recycling of loan funds in the future.

Lessons on Supervision. IFAD requires substantially more involvement in project supervision, if it is to avoid the kind of problems which the project faced when the policy environment underwent substantial changes. The long-distance supervision through a cooperating institution, while useful for handling regular project implementation matters such as loan requests and disbursements, did not function adequately when dealing with more crucial project issues. In addition, there was an insufficient flow of project-related information and reports from IDB to IFAD. As experience with the project demonstrates, certain supervision aspects should not be delegated completely to the cooperating institution, whose orientation may not be the same as the Fund's. In addition, in credit projects, IFAD should ensure that agricultural credit experts participate in supervision missions. Finally, close attention should be paid during project supervision to the further use of credit funds recovered.

Incorporation of lessons learnt in the New Jamaica project

After the completion evaluation of the Small Farmers Credit Project a new project for Jamaica is being designed. The new project is formulated within the framework of an overall structural adjustment programme which emphasizes trade liberalization and an increase in lending rates to the small farm sector. The project design follows the general policy of privatization. Thus, agricultural extension would be subcontracted by the ACB to assist its clients, according to their demands, instead of the public sector supply led approach followed in the SFCP.

The analysis of the target group has been given much more importance in the preparation of the new project than in the SFCP. The target group would include the rural poor (not only the farmers). In addition, the eligibility criteria would include household income instead of being restricted to acreage, and credit ceilings will be established.

Beneficiary participation is intended to be of essential importance in the new project. The ACB/PCB network would become a cooperative banking network. More and more decisions will be transferred from the ACB to the PCBs, which played too modest a role in the SFCP. Moreover, the role of the management committees within the PCBs would be strengthened, therefore allowing for a direct involvement of the rural poor at the decision making level.

The project will not include a specific community soil conservation works component. These issues will be promoted as part of the extension messages. In this way those activities will be integrated with the rest of the project and project coordination will be simplified, allowing thus for greater efficacy and efficiency than in the SFCP.

The new project is based on the development of a sustainable cooperative banking system, introducing savings as a new element to the ACB/PCB system and strengthening the PCBs through appropriate technical assistance and training.

As has been done in the Hillside Farmers Support Project (Loan[No.[217-JA), the monitoring and evaluation functions will be separated. Monitoring will be undertaken by the ACB/PCB system. Most of the indicators would be based on data available within the ACB/PCB information system. On the other hand, evaluation would be the responsibility of the Project Analysis and Monitoring Company. The lack of separation of the monitoring and the evaluation function was a key reason for their lack of appropriate functioning in the SFCP.

As project coordination was one of the major problems in the past, the project would have only one executing agency, the ACB. All other services would be subcontracted via the Project Management Unit.