Ex-post Evaluation

In the 1970s, the Indonesian economy grew robustly as a result of favourable economic policies. Since 1981, however, the fall in oil prices has caused a severe deterioration in the country's external terms of trade. Economic policy measures taken in recent years aimed at achieving economic growth and stability and included restraints on public expenditures, tax reforms, and a more active exchange rate policy.

One of the strategies adopted by the Government of Indonesia (GOI) for spreading economic growth is the opening up of new lands for agricultural production through the Transmigration Programme. Lack of draught power, however, has constrained the cultivation of more lands for food and tree crop production. To overcome this constraint, GOI launched a programme aimed at supplying draught animals to the transmigration areas.

Project design and objectives

The primary objective was to increase agricultural production by overcoming the draught power constraint. The components of the project included phased procurement of 45 000 cattle over a five-year period for distribution in the provinces of Sumatra, establishment of a Project Management Office (PMO), provision of infrastructure for cattle transfer, forage improvement, technical assistance and programme support including research into the Jembrana disease.

Increased food production was envisaged at appraisal as the principal benefit of the project. The major beneficiaries were expected to be about 40 500 smallholders in southern Sumatra. It was assumed that cattle could be procured at reasonable prices and that poor farmers in transmigration areas would be willing and able to receive and care for the animal. Specific, quantified assumptions, were embodied in calculations of the project's economic and financial viability.

IFAD considered the potential socio-economic impact of the project to be sufficient to justify the risks involved. The total project cost was USD 40 million with IFAD financing some 65% of the costs. A Loan Agreement between GOI and IFAD signed in June 1980, provided GOI with a loan of SDR 20 800 000 over a six-year period.

Evaluation

Implementation context

Project Coordination and Management. Coordination by an Inter-Departmental Steering Committee 1and an (intra-departmental) Technical Steering Committee has been adequate and provided high-level policy support. Liaison with other government agencies improved over the life of the project as project procedures and reporting were refined. At the provincial level a similar inter-departmental committee coordinated project activities under the chairmanship of the Governor. Meetings were called only as the need arose, but coordination at provincial level never created serious problems, although lines of functions and authority had to be kept under constant review. There was a general lack of coordination with other livestock distribution programmes in such areas as targets for procurement, studies and investigations on common aspects, and in terms of exchanging implementation experience.

The project was characterized by strong management, which was assisted by a detailed and functional Management Information System (MIS). Initially, the operations of the PMO were impaired by the lack of a proper office, transport and office facilities. When PMO finally moved back into the main DGLS office, this resulted in improved relationships with other DGLS operations and a firmer control over budget planning and finances.

Initially, project activities at the provincial level were to be handled by the regular staff of the DGLS, organized into task forces. However, when it was recognized that these teams needed to be strengthened and brought under the more direct control of the PMO, Provincial Management Units (PMUs) were created as separate entities, thereby strengthening the functional ties of the project with its field personnel. The fact that the project would often be a major component of the activities of the provincial livestock services meant that activities had to be integrated into the provincial livestock administration. This proved to be highly conducive for the sustainability of activities. By the end of the project, PMUs had become one of the most effective implementing arms of the project. Their good relationships with the provincial livestock services and other agencies have contributed to better coordination not only within the livestock sub-sector activities but also with the staff of other government agencies. The PMUs provided the link between the PMO and field staff and between PMO and provincial government authorities. PMU chiefs were mainly functioning as coordinators, but their management role increased when the units were increasingly entrusted with greater responsibility by PMO to handle implementation problems at the provincial level. This enhanced their integration into the regular livestock services and strengthened links with other provincial services.

Small task forces at provincial level with a veterinarian, an animal husbandry officer and a forage agronomist officer plus support staff were established (SATGAS). These task forces operated under the administrative control of the provincial authorities and received technical direction from PMO. The establishment of the SATGAS teams has been very successful. The diversion from the original plan of having district and provincial level field staff, to the concept of SATGAS teams situated in the locations themselves proved sound. Their ability to carry out hands-on management ensured continuity of progress of the project and provided farmers with basic support. The SATGAS played a pivotal role in the operation of an effective MIS.

Whilst not a problem in the early days, the ever increasing number of participants and size of project herds has however implied a disproportional amount of SATGAS time being spent on data collection, administration and bookkeeping. This also led to the slow processing of loan documentation, particularly in peak periods of cattle procurement. SATGAS teams cooperated with extension staff from other departments and especially with the sub-district livestock services, largely on an informal basis.

The level of training received by key farmers is generally high. However, the transmission of information from key farmers to group members has been weak. It is important for the SATGAS to closely supervise the transfer of knowledge to group members and to ensure that this process is functioning effectively.

Project performance was clearly enhanced by the careful screening and selection of staff employed at all levels. Important elements in staff performance have been the honoraria paid by the project over and above normal civil service conditions and adequate provision for such items as housing, travel, vehicles, equipment and training.

Consultancy services played a major role in the project and it can be argued that the overall success of the project is partially attributable to the involvement of national and expatriate consultants. The question is whether there was a trend to diminish the reliance on consultants (national or foreign), and whether this was translated into less consultancy services as project staff learned from the consultants and assimilated the knowledge and skills acquired into their daily activities. This was clearly not the case; in fact the reliance on consultant's services during project implementation was even heavier than expected. It is acknowledged that it is not easy to recruit good Indonesian project management personnel on the basis of relatively low civil service salaries. One solution would be to hire qualified Indonesians as consultants and assign them line responsibilities. Some of the technical assistance used by the project falls into this category.

Civil Works, Vehicles and Equipment. Funds were provided for livestock handling facilities, quality control and observance of quarantine. During implementation, the quality control centres were replaced by holding grounds, to process units of 550 or 1 100 head of cattle. This allowed better handling of large numbers of cattle and provided longer term benefits for the livestock industry. The civil works programme was confronted with considerable delays, especially in the early years as a result of procedural problems in project start-up, difficulties in obtaining suitable sites, GOI's budgeting delays, and hesitation by the provincial authorities because they were not convinced at first about the project's long-term benefits.

Most of the handling facilities were constructed by private contractors and results have been satisfactory. The design of holding grounds was standardized bearing in mind future use and reductions in costs. The mission inspected several of the sites and found that constructions were solid and spacious, with the exception of cattle dips that were judged too small. Locations were not always optimal but it was understood that land availability was a constraint, especially near harbours. It was noted that for some structures specifications may have been excessive, particularly due to their limited post-project use. Maintenance of the structures varied from location to location, with some clearly in need of serious work, whereas others (i.e. on Bali, where frequent use was made of the facility) were well kept.

The project purchased 192 motorcycles and 71 vehicles. Government procurement procedures were often slow and budgeting was not always timely. No funds had been allocated for the replacement of motorcycles, but replacements procured under the second phase (SCDP-II) partly resolved this issue.

Cattle Procurement. A total of 52 410 head of cattle were procured, 16.5 % more than the appraisal target of 45 000. Traders' profit margins did not appear to have been excessive, although a certain amount of overpricing occurred during the first rounds of procurement. Losses and risks during cattle movements were reduced through rigorous rejection of animals which did not meet specifications and through the establishment and improvement of quarantine stations and holding grounds. Nonetheless, a number of weak animals got through to the distribution areas. The inferior quality of breeding cattle is one of the reasons for the late maturity of heifers, which in turn causes late pregnancy, low calving rates and, eventually, delays in loan repayments.

Cattle Distribution. The project, through the credit-in-kind system, facilitated the distribution of 50 773 cattle (12.8% more than the appraisal target of 45 000) to 46 572 farmers in five provinces of Sumatera. High mortality rates blighted cattle distribution in the early years. Most of the distribution sites were upland farming areas, and some of them had poor soil conditions. Adequate soil nutritive status is important to avoid high mortalities linked to soil deficiency.

Credit-in-Kind and Offspring Redistribution. The credit-in-kind system enabled the poor to acquire breeding cattle-cum-draught animal on very favourable terms, and this accounted for the widespread interest and participation in the project. Redistribution of offspring began in 1986. By June 1994, some 46 183 offspring distributed under the project had been returned as in-kind repayments. This represents 47.9 % of the maximum number of offspring (96 467) farmers were expected to return to the project within the maturity period. Factors explaining the low repayment performance include the distribution of poor quality heifers to beneficiaries, the selection of project sites with unfavourable agro-ecological conditions, and poor animal husbandry practiced by the beneficiaries. Loan processing and documentation continued to be a problem area. The attempt to monetize credit-in-kind also created unnecessary complexities.

Credit risks, that is the likelihood of farmers not being able to pay back their loans, arise mainly from cattle risks which themselves result from the forced slaughter or unforeseen death of the cattle not due to negligence or deliberate action of the concerned cattleholder, but rather due to infertility, contagious diseases, accidents, and the like. The procedures for replacement of cattle losses due to these causes were found to be cumbersome and highly centralized.

Jembrana Cattle Disease Research. High-quality work was evidenced in research results. Fully equipped and operational, the Bali Cattle Disease Investigation Unit (BCDIU) in Denpasar determined that the disease is caused by a virus, but it needed further funding support to develop a cheaper vaccine. Continuation of the research activities was made possible through the financing of the second phase of the project. However, the Evaluation Mission questions the validity of financing this type of research through a loan for a development project.

Forage Seed Development. In the procurement areas, the forage seed development scheme worked well in the initial years, but the demand for forage legume seeds declined in latter years, due to which the prospects for the programme dimmed. A salient feature of the scheme was savings mobilization among the participants. This was a practical way of educating village people on the use of bank services. In the distribution areas, the forage component was poorly designed from the outset. It was only towards the end of the project that mixed farming systems were promoted, aimed at improving the sustainability of the dominant cropping component, while securing forage for livestock.

Financial Performance. Counterpart funds in the first phase were not disbursed according to the SAR schedule, due mainly to constraints on the government's budget and rigid budgetary procedures. At project closing, the government's contribution had reached 65% of the level expected at appraisal and 18% of total expenditures. Delays in start-up and inadequate GOI funding also resulted in a slight delay in the use of the IFAD loan, but this was corrected by the end of the second year, since then, loan disbursements have generally been on schedule. A computer based financial accounting system developed by the PMO has greatly assisted in establishing a high standard of financial recording and reporting.

Compliance with Covenants. All supervision missions have reported that the covenants set in the Loan Agreement were met, although audit reports were regularly submitted late.

The Role of the Cooperating Institution and IFAD. The project was intensively supervised by IBRD (on average one supervision mission every 4.8 months). This was in response to the identification of this project as a high risk project. The presence of the country office of the World Bank in Jakarta played an important role in this process but the actual presence of IFAD staff in the supervision missions was very limited.

Institutional Development

Strengthening of Livestock Services. The project organization successfully achieved a high level of effectiveness. Capacity was strengthened during implementation. Organized into a three-tiered structure, the project organization resembled a miniature model of a development agency. The organizational structure was highly conducive to the implementation of a project on this scale. At the helm was the PMO with comprehensive management functions. At the provincial level, a PMU was set up in each distribution province under the technical and operational guidance of the Project Director to implement all project activities. At the field level SATGAS teams organized project and credit activities, provided extension and animal health services to project beneficiaries, and acted as an effective reporting channel to project management. The Transmigration Department recognized the superior institutional framework established by the project with respect to cattle transfer programmes and proposed passing greater responsibility to the PMO for this function. This high level of performance was maintained throughout the follow-up project. The farmers' groups, created with the assistance of SATGAS staff, are also an achievement. However, these groups need to be revitalized and given a permanent function if they are to be sustainable.

Training. IFAD's funds were reallocated, from unused civil work expenditures to staff training overseas and GOI's funds were used primarily for SATGAS, key farmers' training, and for the training of enumerators for Monitoring and Evaluation (M&E) survey work. At PMO and PMU level, staff members received courses in management, project organization, M&E and problem analysis, both in Indonesia and abroad. SATGAS and key farmers received extensive training. In total IDR 344.1 million was used for training out of which IFAD financed IDR 249 million.

The weakness of the livestock extension services in the project areas, recognized at appraisal, has diminished and extension has been strengthened. The question of the sustainability of field support services developed under SCDP I was therefore negated by the implementation of SCDP II.

Monitoring and Evaluation System

Management Information System (MIS). The MIS has been a key factor in assuring the project's capacity to meet physical and financial targets. A comprehensive, computer-based system provided management with necessary information for decision-making. Measuring physical achievements and credit repayment was overemphasized. The system still lacks socio-economic indicators. The practice of deleting a farmer from the system, once his/her credit has been repaid, defeats the purposes of follow-up and post-project evaluations.

Socio-Economic Impact Evaluation. The monitoring group of the PMO comprised three to four consultants plus a team leader and short-term assistance from the technical advisor who would visit the project once or twice yearly, mainly to assess project impact. Surveys were often overdesigned and underanalysed. They did give important data on the use of cattle but the integration of this information into the management's decision-making process was never achieved.

Project achievements and impact

Animal Performance. The calving rates of cattle distributed under this project are not significantly different from those in the procurement areas. However, the mortality rates are lower in the distribution areas. Project data reveal that of the 107 distribution sites, 55 (51.4%) had lower than 50% calving rates, and 25 sites (23.4%) had calving rates of 60% and higher.

Impact in Distribution Areas. The principal project benefits to the farmer are derived from incremental food crop production generated by ploughing. The project's socio-economic studies of 1988 and 1989 reveal, among other things, that the use of cattle has allowed an average increase in the cultivated area of 0.5 to 0.7 ha per farm. The studies also indicate that farmers who plough actually spend less time than non-ploughing farmers on development and cultivation of their uncleared parcels of land. Survey data indicate that the extra income from crop and livestock sales is smaller than the extra income which non-ploughing farmers obtain from off-farm work; nonetheless, ploughing farmers are able to achieve better food security and food intake levels.

After repayment of the first two calves, further offspring serve as an additional source of income. In the use of these offspring, three categories of beneficiaries can be distinguished:

(a) those who use the income earned to diversify their activities, expand their landholdings, purchase goats or chickens, or establish a food stall; for these people, receiving the cow has been a catalyst enabling them to improve their economic situation considerably;

(b) those who regard the cow as a form of savings for the future; they use the income to meet daily needs and rarely, if ever, use the income in developing other activities; and

(c) those who sell their original cow to finance emergency needs such as paying for health care, to finance improvements to the house, or to make a trip back to Java.

Project beneficiaries revealed that significant improvements in their living standards resulted from their involvement in the project. They expressed these improvements in their standard of living as being exemplified by the ability of families to visit their place of origin, meet educational expenses, and make worthwhile improvements to their homes, all of which contribute greatly towards improving social status.

Impact in Procurement Areas. The project's socio-economic study estimates that farmers receive 30-40% of the contract price received by main contractors, and the remainder is distributed between collectors and traders. Collectors and traders do not appear to be making excessive profits, given the costs and risks of supplying cattle to the distribution areas.

Women. Field discussions by the mission revealed that many women were clearly involved in the regular care of the cow but saw their role as assisting, rather than managing. When extra labour was required for collecting fodder, where the children were old enough, they were more likely than the wives to be called upon to help.

Environmental Impact. Following the movement of the cattle, there are a number of stages of interaction with the environment of which the following is a non-exhaustive list of the risks and benefits involved, notably:

(a) the impact of grazing patterns on soil erosion; at appraisal it was argued that existing grazing patterns lead to soil erosion and fertility decline in South Sulawesi, Nusa Tenggara Barat (NTB) and Nusa Tenggara Timur (NTT) and that this would diminish with the procurement of cattle from those areas; the mission found no evidence that this had actually occurred.

(b) cattle faeces and urine pose the greatest danger for the environment during transshipment when very large numbers of animals are brought onto the holding grounds and quarantine stations; although the situation of effluent disposal from these sites has greatly improved, some sites are still not equipped with septic tanks, while other sites still require upgrading.

(c) greening of the land areas with fodder trees or forage counteracted any potential ecological damage caused by the felling of tropical forests for land settlement; this is clearly a positive contribution of the project;

(d) regulations regarding the placing of stables were not always followed; the mission was frequently confronted with a situation in which stables were built too close to a well, which could have adverse effects on family health;

(e) the use of manure, collected at handling facilities does benefit some farmers, but this should be more actively promoted; almost all farmers who received cattle apply the manure to their land; and

(f) ploughing carries the risk of intensifying the use of land and thereby increasing the risk of exhausting soil fertility; in order to compensate the soil for fertility depletion, farmers should be taught to use fertilizers in recommended dosages.

Research into Jembrana (Cattle) Disease. Evidence was obtained that Jembrana Disease is caused by a virus belonging to the family Retroviridae. Although much progress had been achieved, additional finance from the IFAD follow-up project was needed to further the research done to date, and even at the end of the second phase, more financing is still needed, indicating the difficulty of including a research activity without a clear time-horizon in a development project with a five-to-seven-year implementation period.

Economic Rate of Return. Calculations based on recent herd parameters suggest an economic rate of return on the project 16.2%.

Main issues and recommendations

Acquisition of Draught Animals. The project enabled the poor to acquire breeding cattle-cum-draught animals without imposing on them the burden of generating substantial cash incomes for the repayment of loans. This accounted for the widespread interest and participation of beneficiaries in the project. The project also served as a model for an extended follow-up project, which IBRD was then interested in cofinancing. Availability of cattle led to the opening up of more land to agricultural production. The area of land cultivated by farmers generally increased on average by 0.25 ha with the use of cattle for ploughing. As a result, household earnings from agricultural production increased for most of the beneficiaries.

Factors Influencing Project Performance. One of the factors contributing to the project's success was the high level of government support for the project. This was possible because the project's objectives dovetailed with those of the government for the development of transmigration areas. Most of the project directors have proven to be capable managers. Theirs' was a centralized management approach buttressed by their ability to sympathize with project staff and to appreciate their hard work and labour. Most of the project sites selected for cattle distribution were transmigration areas, food crop production from which was constrained by the lack of draught animals. There were project sites, however, with harsh agro-ecological conditions and the cattle in these areas generally showed poor health and low calving performance. During the project period, local prices of cattle steadily increased, thus giving beneficiaries brighter hopes for future cattle sales and continuing inspiration to care for their cattle.

Project management was able to delineate early in sufficient time the best ways of organizing the project. The project made it possible for the weaknesses of DGLS, recognized at appraisal, to diminish and for its extension capacity to be strengthened. DGLS, a well-knit organization, was established to operationalize a credit-in-kind system rooted in a tradition that continues to be socially acceptable and widely practised. MIS turned out to be the right tool for management to control nation-wide operations. Adaptation of MIS to the district level (SIMKAB) and provincial level (SIMPRO) is still in its infancy and needs considerable attention before these would be institutionalized. The project underscored the importance of improving the quality of project staff and key farmers by organizing training courses for them. Discipline in the organization was reinforced by an above normal honorarium paid by the project and by adequate provision having been made for housing, travel, vehicles, equipment and training - all of which contributed to high morale among project staff and their willingness to handle difficult situations and to produce good results.

Project support for civil works contributed to a great extent in minimizing the risks that, at appraisal, were feared to be serious. Although delays in the start-up of Phase I and the inadequacy of GOI funding resulted in a slight delay in the use of the IFAD loan, this was corrected by the end of the second year, from which time loan disbursements were largely on schedule. The PMO developed a computer-based financial accounting system which has greatly assisted in establishing a high standard of financial recording and reporting. Project support for research has also enabled local scientists to identify the virus that causes the Jembrana cattle disease and to devise a strategy for its control.

Sustainability. Six years after project closing, it has been clearly proved that DGLS can operate the credit herd as a self-perpetuating, revolving commodity credit facility. In the long term, DGLS should devote more efforts and resources towards creating and maintaining a herd of high quality breeders, and to entrust the redistribution of offspring to institutions operating as "credit intermediaries" at the village level (e.g village cooperatives, NGOs, rural banks, or even farmers' groups). This course of action would benefit from a decentralized organizational set up. In addition, sharing of the proceeds from the sale of culled bulls should be restructured to favour to a greater extent the provincial livestock services.

Lessons learned

Lessons Learnt. It appears that on many occasions lessons learnt during the first phase provided only part of the picture and with ongoing experience under the second phase, these lessons had to be reconsidered. Ideally, when experience is gained through the implementation of a project, lessons learnt from this have to be analyzed in the light of the experience of other projects and of the general body of knowledge that exists about the technical, economic and social content of similar situations. Hence, collaboration with local or national research institutes could be envisaged. When a lesson emerges from an evaluation, a review or a survey, there is a tendency to translate this lesson directly into a particular recommendation. This is justified in the case of an ongoing project, where management needs guidance for the improvement of project implementation. However, in the longer term, it is more interesting for governments and development agencies to supplement existing information and to develop models with wider replication potential, using a holistic approach.