Interim evaluation

Project design and objectives

Target group

In the appraisal document (1990), the target group was estimated at 21 000 farmers having holdings of less than four hectares; 77% of these farmers owned their property. By focusing on this group, the project sought to benefit some 8 000 smallholders directly (38% of the target group), irrespective of their land tenure arrangements. The annual income per family was estimated at USD 560. Some 7 000 farmers were to receive credit services, and approximately 2 000 women would benefit directly from technical assistance and loans for agricultural production, microenterprises and home improvements.

The post-evaluation mission, which visited the country in April 1993, brought back updated and better quality data on the project area and, as a result, the expected number of direct beneficiaries was reduced from 8 000 to 7 000. Based on the findings of the February 1994 baseline study and experience gained in the region, the number was subsequently reduced to 6 000 direct beneficiaries, i.e., 16% of the 38 000 farmers having less than four hectares who were covered in the baseline study. Of these farmers, only 34% owned the land they worked, while 60% were tenants.

Objectives and components

The project’s objectives are to improve the quality of life, income levels and nutritional status of small farmers in the project area, with special attention being paid to women. The project also seeks to contribute to social stability in the area by creating long-term employment options, promoting agricultural diversification towards more productive patterns, and providing solutions to the critical situation of natural-resource conservation.

The project has eight components: (i) Credit services, for approximately 7 000 farmers, including an estimated 1 500 women. Project funds will go towards covering the costs of agricultural and forestry production, development of small agro-industries, construction and operation of 12 rural marketing centres, minor housing and service improvements, and support for works and services for rural communities by way of non-reimbursable social assistance. (ii) Extension, consisting of two subcomponents: (a) technology validation, to diversify production, improve the food supply and generate surpluses, while conserving natural resources; and (b) rural extension, to strengthen and provide resources to the extension services of the Ministry of Agriculture so that they in turn can train beneficiaries in technologies that will allow them to increase the yields of basic grain crops and incorporate new ones, such as vegetables and fruit, while ensuring natural-resource conservation. (iii) Support for Women’s Participation, to ensure their active involvement in development and the improvement of their living conditions. (iv) Training, for project staff and farmers identified as leaders. (v) Marketing, including support for the formalization of the Rural Centres for Agricultural Marketing and Inputs (CERCAIs) and training for partners in various marketing aspects. (vi) Technical Assistance, to provide specific support for project activities. (vii) Project Executing Unit, which would be set up and assigned the necessary premises, equipment and personnel. (viii) Monitoring and Evaluation, to lend support to the Ministry of Agriculture and the Project Executing Unit.

Although the components remained the same throughout the life of the project, in some cases the institutions responsible for their execution changed (as a result of reforms in El Salvador’s public sector) or their orientation and content were changed.

Expected effects and assumptions

PRODAP, like other projects formulated at the time, is based on the three elements of credit, validation/transfer and marketing. The basic assumption is that the available technology exists but must be validated for small farmers in the project area and distributed through a specific extension system that is limited to the target population. Since the project includes a credit component, beneficiaries would face no financial constraints to adopting these more advanced technologies. The project’s marketing component should improve margins, and thus revenues, for beneficiary farmers.

With regard to improving the production levels, productivity and operating margins of these farm units, it is assumed that the farmers use little or no improved seed, agrochemicals or fertilisers, that they do not have collection facilities for basic grains and that they do not have access to formal credit from the Agricultural Development Bank (Banco de Fomento Agrícola, BFA).

Regarding the training and women’s participation components, the assumption is that these components have an impact on all the others and will improve the efficiency and effectiveness of the project in meeting its targets and achieving its goals.

Other general assumptions implicit in the design are that the data available at the time of project formulation reflected the actual situation in the departments of San Vicente and Cabañas; that the national institutions would retain their same structure and functions; and that the project would focus on agricultural development without making allowances for the high population density in the project area (population density was 171 and 184 residents per square kilometre in Cabañas and San Vicente respectively - Appraisal Mission report 0228-ES, paragraph 107, June 1990), the average size of rural families and the scarce land available to each family for agricultural use.

The project’s expected effects were as follows: (i) approximately 7 000 beneficiaries (including 1 500 women) would gain access to formal credit, and they would adopt technologies recommended by the project for planting new crops with high market value as part of the diversified production generated by the technology transferred; (ii) traditional crops would have higher productivities; (iii) farmers would introduce new agricultural systems that have known beneficial effects on the improvement and conservation of soil and water resources and forest species; (iv) production-oriented and service microenterprises would be set up and operated by project beneficiaries; (v) marketing processes would be based on small centres and would be more efficient; (vi) support services would be strengthened for generating, validating and transferring technology, credit, training and marketing; (vii) women would be more involved in project activities and would enjoy equal opportunities as beneficiaries; and (viii) suitable arrangements would be established and consolidated for project execution and inter-institutional coordination.

These assumptions and effects were revised to reflect changes in the local setting subsequent to appraisal (June 1990), loan signature (March 1992), loan effectiveness (November 1992) and project start-up (December 1992 to March 1993). Based on an analysis of the project’s status and local developments (i.e., the signing of the Peace Accords and the launching of institutional reform), a post-evaluation mission (April 1993) recommended a series of adjustments that were to be made to the project and the annexes to the loan agreement.

The findings of the February 1994 baseline study diverged significantly from the values registered at appraisal for the principal variables and indicators, which had provided the justification for the project actions as designed. This meant that some of the basic assumptions would need to be rethought entirely.

The baseline study also showed considerable divergence from the values used at appraisal with respect to agricultural output, the use of improved seeds, fertilisers and insecticides, livestock production, and plant and equipment. In addition, the study furnished information on the target group’s access to credit and extension services and a breakdown of their income by source; it also pointed up some significant differences between the two reports with regard to per capita income (expressed in current dollars).

The body of the report shows the values that were used at appraisal and those that were used in the baseline study.

Evaluation

Implementation context

Social and economic developments in El Salvador have had differing effects on project execution and its results.

General circumstances resulted in a series of changes within the Project Executing Unit, stemming mainly from institutional reforms that were introduced during the implementation period. As a consequence, the unit took over direct responsibility for some components that had not been assigned to it originally.

These important changes in the project’s organization and operating arrangements arose as internal responses to real or perceived changes in the setting in which PRODAP was being carried out, and to the effects of weather on the productive activities promoted by PRODAP.

Project achievements

General Achievements

The action taken by the various departments in carrying out the project’s components, coupled with the work of the Project Executing Unit’s technical staff, has allowed PRODAP to amass a substantial pool of achievements, which can be expressed numerically as a function of the project’s targets.

The project provided technical and financial assistance to over 5 000 beneficiaries, thus meeting 85% of its target of 6 000 beneficiaries. Of the total USD 8 million allocated for credit services, USD 5.3 million (equivalent to 66% of the target) was placed. Lastly, the five years of actual project implementation that have elapsed thus far represent 83% of the six years called for in the appraisal document.

One particularly noteworthy achievement is in the area of community works. The USD 150 000 provided under the project to finance 100 community works on a non-reimbursable basis was able to mobilize from beneficiaries and other sources an average of USD 2.33 for each non-reimbursable dollar, thanks to the management provided by the individual organizations and committees.

Ways of Enhancing the Project’s Achievements

During its final phase, PRODAP should reassess its targets for efficiency and effectiveness, since it might be wiser to consolidate the project’s current achievements than to try to increase its coverage.

The process of selecting strategies before activities are launched is an area that needs to be strengthened. Ideas need to be exchanged prior to analysing and harmonizing the targets set in each component’s annual plan of work; this is a necessary step that cannot be skipped if the current achievements are to be built upon. Furthermore, recommending that this activity be included at the end of the project implementation period is a way of testing innovations that might make for better execution in future projects; in this connection, PRODAP should serve to verify methods that will yield higher levels of efficiency and effectiveness.

The project should focus on boosting the loan-recovery rate and preventing any increase in the default ratio.

Effects, assessment and sustainability

Effects and Assessment

(a) Effects and Assessment of the Project’s Objectives

The PRODAP project was launched in the midst of the 1992 Peace Accords against a backdrop of turbulence and political instability that clearly affected initial actions and decisions under the project. From the viewpoint of the national authorities, the project provided a quick and efficient government response for the rural population that had been affected by armed conflict in the departments of Cabañas and San Vicente.

However, the project’s impact was limited by external developments that could not be foreseen or anticipated by the project. The agriculture sector saw sharp declines in real prices for products that could be sold on domestic markets; this lowered farmers’ income levels and the sector’s profitability. Family remittances had fueled exchange-rate appreciation since 1991 (with the rate dropping 7.5% a year between 1990 and 1995), and they were a determining factor in the decrease in domestic real prices, especially farmers’ prices, during the period 1990-97. In real terms, the 1995 prices for maize and beans respectively were 34% and 20% lower than the base year of 1990. At the same time, imported inputs (e.g., fertilisers and agrochemicals) became relatively more expensive. This situation still exists today.

The Interim Evaluation Mission produced no quantitative evidence to describe the situation of project beneficiaries and the project area in general in a non-project scenario. It did reveal, however, that residents of the two departments felt they would be worse off without this project. The beneficiaries interviewed all acknowledged that PRODAP was the first development project carried out in the two departments and that the services were suited for both.

The project clearly contributed to the objective of social stability in the two departments, thanks to the equal access enjoyed by men and women to the project’s services. PRODAP also provided and put in motion a series of unexpected support services and activities for the target population (e.g , credit, extension, support for women, training and marketing).

As a result of these activities, living conditions improved noticeably for a number of project beneficiaries, i.e., the leading producers (both men and women).

Support for livestock production (dairy cows, hens and, more recently, goats) helped to improve the diet of many beneficiaries and their families. Project support for family gardens may also have contributed in this regard.

Actions in support of women not only attained substantial coverage levels but also were instrumental in families’ acceptance of the gender approach. The general atmosphere was one that fostered equitable relations between men and women, especially at the family level.

A major impact was also felt in the level of community organization for the management and execution of community works (e.g., health clinics, schools, roads, pedestrian overpasses). This was achieved by using the grant funds provided under the project to mobilize non-project resources from the community as well as from public and private-sector sources. For the 95 community works that were completed, the project’s grant funds represented only 30% of the total cost of USD 470 000.

PRODAP has had a relatively low impact in terms of boosting basic-grain yields. This is owed to the fact that farmers were already using improved technologies for these crops and to the fact that actual yields were higher than those assumed at appraisal.

As for the diversification of production, basic grains (maize, sorghum and beans) continue to account for most of the cultivated surface in the project area, although several new crops have been introduced. After the farmers become familiar with the production and marketing of these crops, the operations could be upscaled and would have a major positive impact throughout the year on the level and distribution of income derived from the sale of agricultural products.

Although the target group is familiar with techniques for soil conservation and fertility management and many of them have adopted environmentally sound practices, the areas involved are too small to have any appreciable impact. As these practices are implemented in more areas, the impact will be greater. To consolidate these actions and their widespread application, PRODAP should focus on these activities up through closing with a view towards their sustainability.

One of the major impacts in terms of soil management has been the adoption of the "no burning policy"; this is a mandatory condition (beneficiaries must sign an "environmental commitment" whereby they agree not to burn stubble fields as long as they have PRODAP beneficiary status) in order to qualify for the PRODAP project. Residents acknowledge that, as a result of the project, beneficiaries and non-beneficiaries alike have stopped burning stubble fields and soil quality has benefited.

There has been no identifiable impact in terms of natural-resource management or solutions to the problems of low fertility (caused by the lack of biological regeneration) and water management in the face of the soil’s dwindling moisture-retention capacity.

(b) Impact Perceived by the Beneficiaries

Generally speaking, the beneficiaries rank the services and benefits generated by the project as "good" to "very good". They feel that living conditions have improved as a result of the project, thanks to its support in building self-esteem, community organizational capacity, interpersonal relations in the family and, in general, making them more knowledgeable so that they can solve problems on their own.

As concerns improvement of income levels, views varied. A small number of farmers felt that their income had increased; others reported an increase in their wealth (capitalization) but not their cash income; many, however, believed they were in about the same position or even worse off than five years ago.

The beneficiaries valued the equal access given to credit for both men and women and were satisfied with the service quality. The high default ratio was ascribed to drought, low market prices, high costs for inputs and, in some cases, poor administration of funds by the users.

The credit that was used for purchasing cattle was felt to have generated many benefits (i.e., milk for the family and community, inputs for sale). Some farmers, though, had fallen into debt and were encountering difficulties because their animals had been stolen or died or because they had very limited land and pasture available. Furthermore, livestock prices have fallen nearly 50% since the time the animals were purchased.

The beneficiaries viewed equal-opportunity access for men and women to project services as a good thing. They also acknowledge the important impact of the true change in attitudes that took place, not only between men and women but also at the level of family relations, in the form of a new, equitable distribution of tasks and responsibilities.

Another important impact was that produced by the project’s comprehensive family assistance and PRODAP’s desire to serve as a facilitator for access to educational services (e.g., adult literacy), health care (e.g., gynaecological examinations, eye examinations, distribution of glasses), and other services.

The construction of community works was also highly valued, because it mobilized both the will and resources of the project and the community (as well as third parties) to either upgrade or build new community works of interest to the majority of the population. The main impact was to demonstrate that, through community organization, results could be obtained more quickly and easily.

Another major impact the beneficiaries signalled was that the extension services allowed them to introduce techniques that reduced cash outlays while protecting the environment (e.g., pest management, semi-organic pesticides, organic fertilisers). Beneficiaries saw that, in some cases, the diversification promoted by the credit and technical assistance components indeed generated benefits (i.e., by offsetting low grain prices, reducing risks and bringing in quick revenue).

Few benefits were perceived in the area of marketing services, aside from the efforts to organize groups for purchasing inputs and, more recently (December 1997), the creation of farmers’ markets.

Beneficiaries also recognized that their skills had improved as a result of the training provided under the project. The 25 farmers who took part in the workshop attended between nine and 17 training events, and in most cases they ended up applying or using virtually everything they had learned.

(c) Impact Perceived by PRODAP’s Technical and Management Staff

In the view of the project’s technical and management staff, PRODAP was able to produce an impact thanks to the actions that allowed them to serve rural residents in the catchment area of the 12 extension agencies located in the departments of Cabañas and San Vicente.

Through these actions, they lent support for: (i) access to credit for more than 5 000 beneficiaries (including 700 women); (ii) organization for the management and mobilization of resources from various sources (e.g., PRODAP, communities, municipal governments, governmental agencies) to improve or construct works in the community interest; (iii) access to project services for both men and women; (iv) a high level of adoption of technologies (botanical and organic pesticides, preventive veterinary care, soil conservation, agroforestry); (v) the diversification of production; (vi) training in production and social issues; (vii) heightened awareness of gender issues, including a change of attitudes in many families; and (viii) organization to ensure access to health, educational and other services.

(d) Impact Perceived by the Local Authorities and Other Public-Sector Agencies

Local Authorities

According to statements made by the local authorities and representatives of public-sector agencies working in the area, the project had a major impact mostly through the training it offered and its involvement with community organizations, which generated trust in the project. The promotion and training activities for women, emphasizing the important role women play in the field, and the strengthening of smallholders through credit services were considered especially relevant project achievements.

One of the initiatives undertaken to consolidate the project’s activities has been the training of "lead farmers" (innovators), who will continue to receive technical assistance after the project closes. These leaders need to be well trained and funds should be made available to ensure that they will continue to receive technical support. With regard to the credit services, it has been proposed that local credit organizations be set up. Farmers and the members of the organizations that have been asked to participate in forming these new specialized credit agencies have expressed reservations, however, citing the risk associated with taking on additional responsibilities in handling funds. Another aspect that was said to need improvement is PRODAP’s coordination with local organizations, in order to provide more support and expand the scope of the project.

Public-sector Agencies in the Project Area

PRODAP has generated replicable work methods that could be useful to other institutions. As an example, the Adult Literacy and Basic Education Programme (sponsored jointly by the Government of El Salvador, Spain, and the Organization of American States) reduced adult illiteracy by five to ten percent in some cantons in Cabañas.

PRODAP’s cooperation with programmes of other government agencies was acknowledged in such projects as the "healthy schools" and "healthy markets" initiatives, midwife training, gender training for health promoters, the construction of sanitation works, and health education for women (cytology examinations).

PRODAP made an especially positive contribution to the development of community organizations. Not only did it motivate communities to increase public participation through the small projects that it financed, but it also obliged other agencies to better train their staff to work with these organizations and to respond more efficiently to community demand.

2. Sustainability and How to Ensure It

The three scenarios described below outline different avenues for consolidating the project’s activities and ensuring a reasonable level of sustainability of the project’s impact during the post-loan period. Their feasibility will hinge on the decisions made as to the action to be taken by the Project Executing Unit, the Ministry of Agriculture, and IFAD/BCIE.

Since the scenarios cover different time horizons, a few words should first be said about some features that will have a bearing on the outcomes.

The strategy decided on at appraisal was that of working with beneficiaries on an individual basis. As the project draws to a close, however, this is producing complications. Although the context changed considerably during the implementation period, no allowance had been made for PRODAP to reassess the situation midway through the project (i.e., a mid-term evaluation). It was thus not able to rechannel its action towards strengthening community groups or associations, which would have helped to ensure the sustainability of the project’s impact and accomplishments.

Furthermore (and this happens commonly during project formulation), no plans had been made for the final stage and closing. During this stage, project staff are more concerned about finding new work than thinking about and planning ways to carry out the activities called for in the annual plan of work. There is also the added complication that usually (and here PRODAP is no exception to the rule) the number of beneficiaries to be added during the last year is higher, in order to make sure that the initial targets are met by the time the project closes.

The situation generally encountered in this phase (as confirmed by PRODAP) is that staff are faced with the dilemma of either seeking new work before the project closes (which means the project staff slowly dwindles) or increasing their workload in order to add new beneficiaries in line with the project’s targets and consolidate the project’s current achievements so as to ensure a higher level of sustainability. Most people choose the first option, since the second offers no immediate payoff and, once the project closes, staff are let go whether or not the work has been satisfactorily completed.

Depending on how many people leave the project because they have already found a new job, adjustments have to be made (sometimes extensively) in the planned level of service for beneficiaries. Tasks are reassigned to those remaining, who must now serve more people than had originally been planned for them.

PRODAP’s Project Executing Unit, which has approximately 110 employees, will have to downsize gradually, not only to lessen the impact of closing in personnel terms but also to reduce staff costs – that might seem excessive – during the final phase of the project, i.e., the point at which it should seek a transition to a more sustainable model.

To mitigate the impact of these events on the project’s sustainability, every effort needs to be made to ensure that actions during the final phase are of high quality and are selected carefully with an eye to the level of sustainability intended for each component, while keeping in mind the gradual reduction that will take place in the Project Executing Unit.

(a) Scenario: Project Activities End on 31 December 1998 and Project Closes on 31 March 1999

Under this scenario, PRODAP would close with a reasonable level of execution of the proceeds from IFAD Loan 267-ES and would have used all of the funds from the BCIE loan. The degree of target attainment would be good to very good. However, in terms of agricultural development and sustainability of the project’s impact, PRODAP would not have time to consolidate its actions, and an ex post evaluation would only find scarce traces of impacts that could be attributed to this project.

The following paragraphs outline the mission’s assessment of the sustainability of the project’s impact under this scenario.

Sustainable impacts and how to ensure continued sustainability

Extension activities should focus on: (i) adjusting local production models and consolidating the rural training centres so they can continue to disseminate technology in the communities; and (ii) ensuring that technical assistance targets mainly those farmers at medium to high risk of defaulting on their loans, and then those who, although they have defaulted, have shown interest in paying and, with technical support, could increase their ability to generate the necessary revenue to pay off their debt.

The credit activities should seek to strengthen the trust fund to ensure its sustainability. Accordingly, priority should be given to recovering loans and reducing the currently high delinquency rate. The BFA has shown no evidence of intent or actions to initiate loan-recovery procedures, and the trust agreement does not empower PRODAP to demand that the BFA take action in this direction.

The default ratio is on the rise, which means that all A and B beneficiaries have probably been reached and that the project should avoid granting new loans to the higher risk (C and D) beneficiaries. Under these circumstances, the first decision should be to clean up the portfolio rather than worry about increasing the number of new customers, who, as mentioned above, are in a higher risk category.

PRODAP has proposed setting up a process to create or strengthen organizations that could serve as credit intermediaries to channel and monitor credit services after 1999. This proposal should be evaluated carefully. It would probably be viable only if these intermediaries have proven experience in administering rural credit through local organizations.

To enhance the trust fund’s sustainability – given the possibility that debts owed to the BFA may be written off (and even if this risk did not exist) – , other financial intermediaries present in the area should be signed on as quickly as possible to participate in administration of the fund, e.g., non-governmental organizations (NGOs), cooperatives, etc.

In order to ensure the sustainability of the identified impacts of the gender approach that was implemented through the activities of the Department of Support for Women’s Participation and the important – albeit less publicized – advances in the consolidation of organized groups (e.g., community development associations), efforts should focus on existing groups and specifically on those in a position to complete works prior to November 1998 using PRODAP community development funds as seed money.

The 1998 Budget Act channelled significant flows of funds through the country’s municipal governments. PRODAP, mainly by way of the Department of Support for Women’s Participation, has had very good relations with this level of government in carrying out community works. These projects tapped existing organizations to consolidate, select and prioritize the works, as well as to manage and mobilize the rest of the necessary funding, which came for the most part from the municipal governments.

To consolidate the impact of the project’s gender and organization achievements and with a longer-term view towards sustainability, activities should be focused on two lines of action: one to assist the municipal governments in identifying communities that can efficiently use the funds channelled under the Budget Act; and the other to prioritize collaboration and support for communities that meet the above conditions in order to assist them in strengthening their organization, in identifying priorities for mobilizing funds to complement those provided by the municipal governments and other sources, and in procedures for purchasing, cost control and supervision of works during the implementation period.

With respect to the training activities, the high level of skills and practices adopted by project beneficiaries allows for optimism in terms of sustainability. Training still needs to be provided to the Extension Department’s technical staff in: (i) financial and economic analysis of farm units, so they may have better tools for choosing models to consolidate the rural training centres, and (ii) managing and analysing credit approval and recovery plans. The evaluations provided by the trainees will be used to identify the technical staff members who will be placed in charge of consolidating the training centres and those who will oversee credit services, particularly recovery. This involves a change in policy in that technical assistance will be offered to farmers who are delinquent owing to low production or prices, but show a clear desire to pay.

Project Impacts that will be Difficult to Sustain

The project’s training activities are not seen as sustainable once the project closes. During the remaining period, options should be sought and analysed to make the Guacotecti Training Centre and the Centre for Less Common Species self-sustaining (for instance, by offering training for non-farm employment, such as carpentry, bricklaying, painting, etc.).

Under this scenario, farmer-training activities, the community development associations and other forms of organization are also seen as unsustainable after the end of the implementation period.

The activities of the marketing and microenterprise component are not considered to be sustainable if the project were to be closed at the end of 1998. The periods for the gestation and consolidation of the processes launched under the component (e.g., the farmers’ market) greatly exceed the remaining time.

Additional Action that is Required

Budgetary funds need to be allotted and ensured for staff who will be working on the closing of the project between 1 January and 31 March 1999.

The trust agreement with the BFA needs to be amended to allow the bank to act as a second-tier agency and thus channel resources to financial intermediaries. This might also enhance the bank’s role in approving loans, since its own demand would be met to the extent that it would have to assume greater responsibility for approving and recovering credit. The amendments to the agreement should establish the arrangements and resources for hiring such intermediaries. Careful study should be given to the feasibility of hiring other financial intermediaries present in the project area. In this case, BFA’s administration of the trust fund would be as first-tier bank (working directly with farmers) as well as a second-tier institution, if it is found to be feasible to work with financial intermediaries.

The following services need to be selected and hired: (i) strategic planning advisory services for PRODAP’s specialists, and (ii) training in analysis of the financial and technical feasibility of production models in credit and non-credit scenarios.

(b) Scenario: Remaining Funds are Used by 31 December 1999 and Project Closes on 31 March 2000

Under this scenario, PRODAP would execute nearly 518 of the proceeds from IFAD loan 267-ES; the BCIE loan proceeds would be executed in full, as would the rest of the funds from the Government. The level of target attainment would be much higher and, in terms of agricultural development and sustainability of the project’s impact, the restructured PRODAP with its new strategy orientation would be in a position to modify and consolidate its action in order to reach a level of impact sustainability that an ex post evaluation would clearly attribute to this project.

The mission’s assessment of the major changes that need to be made in order to ensure the sustainability of impact under this scenario is outlined below.

Sustainable Impacts and How to Ensure Continued Sustainability

Over the longer term, extension activities should be separate from credit activities.

As in the first scenario, extension activities should focus on adjusting the production models and consolidating the rural training centres so they can continue disseminating technology to communities and also reinitiate the process of identifying "lead farmers"; the latter activity would be performed in line with the strategic options chosen and with the technical and financial feasibility analyses that are to be conducted for the various models that will be replicated in the project area. The technical teams that will assist the training centres and the farmers should be limited to three people per team (each specialist would be responsible for 10 rural training centres and 30 farmers); this will help to ensure good service and uniformity in the application of criteria. The teams will receive input from demand analyses (broken down by category, variety, quality and price) conducted by the marketing department and from a financial, volume and quality analysis of production conducted by the planning and monitoring department.

Technical assistance (as in the first scenario) should target mainly those farmers who are at medium to high risk of defaulting, and then those who are already delinquent – and are therefore no longer receiving services – but have shown an interest in paying and, with technical support, could increase their ability to generate the necessary revenue to pay off their debt.

A new credit unit, staffed with specially trained personnel, would be in charge of analysing and approving credit plans drawn up by the extension workers for new beneficiaries as well as additional credit requests from current beneficiaries. As in the first scenario, the objective of the credit activities should be to strengthen the trust fund in such a way as to ensure its sustainability, which means minimizing the default risk of new operations.

The credit unit should assist extension workers in recovering loans so as to pare back the mounting level of defaults, which is jeopardizing the fund’s sustainability. This would represent a major shift in PRODAP’s approach to delinquency and its impact on sustainability.

The credit unit should receive specialized technical assistance for negotiating with and for monitoring and evaluating the performance of financial intermediaries.

Under this scenario, other organizations (e.g., NGOs, cooperatives, etc.) could play a significant role alongside the BFA in channelling credit to users. The credit unit should therefore coordinate support actions for both the organizations and the beneficiaries.

Launching a system to funnel credit through various channels could boost the trust’s sustainability. By involving other organizations that are able to channel credit in the communities, the project would be building a specialized local organizational base that could eventually co-administer the trust with the Ministry of Agriculture.

In order to consolidate over the long term the sustainability of the project’s impact in the gender area and its support for organizations, the Department of Support for Women’s Participation should focus mainly on increasing support for community organization with an eye to fostering families’ acceptance and "ownership" of the gender approach.

Support needs to be provided on two levels: community organizations need to receive support for receiving funds from the municipal government and other sources, while the municipal governments and other sources need to receive support for channelling funds towards selected community works that respond to identified demand. This liaison and management support will strengthen the organizations and help to make them truly self-managing in the future.

As in the previous scenario, the direct actions provided under the training component are not expected to be sustainable once the project closes. However, the training already provided has generated installed capacity among the beneficiary population, who have acknowledged the component’s impact and are putting a good deal of the training received into practice.

Under this scenario, however, training activities should respond to the new requirements of the Project Executing Unit’s staff. In 1999, training should be provided to: (1) Extension Department specialists, in: (a) financial and economic analysis of farm units, so they may have better tools for choosing models to consolidate the rural training centres; (b) working with "lead farmers"; and (c) technical support for farmers at high risk of default or who are already in default; (2) the credit unit’s staff, in managing and analysing credit approval and recovery plans; (3) the marketing department’s staff, in analysis of margins by category, variety and quality of selected products; and (4) the staff of the planning and monitoring department, in monitoring the work of the rural training centres and the "lead farmers" with an eye to giving feedback to the system.

In the longer term, which is the time horizon of this scenario, the Guacotecti Training Centre and the Centre for Less Common Species will have self-management capacity or ensured sustainability thanks to funding from other organizations or projects.

Project Impacts that will be Difficult to Sustain

The training activities – mainly those provided by PRODAP for farmers, community development associations and other forms of organization – are not considered to be sustainable after the project ends. However, with an additional two years for carrying them out, the currently intense levels should be gradually reduced and the activities considered most necessary should be assigned priority.

The achievements resulting from the marketing and microenterprise component are not considered to be sustainable, even with a longer execution period. The training provided and the lessons learned from the experience with the beneficiaries cannot be easily generalized and have very low group and individual sustainability.

Additional Action that is Required

If the Government chooses this option as a way of substantially improving the sustainability of the project’s impacts, it must negotiate with IFAD on how to extend PRODAP’s implementation period until March 2000, in order to be able to use the non-reimbursable funds, matching funds, returns on temporary investments and the revenue from sales and services of the Centre for Less Common Species.

If IFAD maintains 31 December 1999 as the mandatory project closing date without the development objectives and a reasonable degree of sustainability having been achieved, the Government will have to allocate the necessary budgetary funds to allow for organized closing of the project during the period from 1 January 1999 to 31 March 2000.

In both of these scenarios, the trust agreement with the BFA would have to be amended and fund-administration agreements would need to be drawn up with other financial intermediaries present in the project area.

The consulting and training called for in this scenario would be in addition to those indicated in the previous scenario.

The Project Executing Unit would also need to undergo various changes (e.g., creation of a credit unit, reassigning of functions to the Extension Department, etc.), with a gradual reduction in the overall number of its staff, depending on the transition strategy adopted.

(c) Scenario: A New IFAD Project in the Year 2000, to which PRODAP would

Contribute Lessons from its Experience

Two projects are currently under way in El Salvador with IFAD financing (PRODAP and the Rehabilitation and Development for War-Torn Areas in the Department of Chalatenango [PROCHALATE]). A new one, the Rural Development Project for the North-Eastern Region (PRODERNOR), is about to begin. Considering the geographic distribution of these projects (i.e., the departments of Chalatenango, Cabañas, San Vicente, Morazán and La Unión), there is a large area in the eastern region in which new development projects could be initiated; the area has high poverty levels and was one of those hardest hit during the war. It would be more important, though, to consolidate and apply experiences from the three aforementioned projects.

This scenario assumes that a new IFAD-financed project will be designed for the eastern region and will begin in mid-2000.

Sustainable Impacts

Under this scenario, PRODAP would have closed with clear signs of the sustainability of the project’s impact and it would have generated rich experience in strategies and methods for activities and actions. Two PRODAP outcomes in particular would be instrumental in maintaining development momentum in the region.

First, the managerial, administrative and technical staff from each of the various components/departments/units would have gained valuable experience in implementing projects aimed at mitigating the conditions of rural poverty and would have developed abilities and changed attitudes. These are the lessons that could be learned from this process and that constitute PRODAP’s impact on the group of persons who, at different levels of responsibility, have pursued the project’s actions to achieve the proposed objectives.

These actions would be sustainable only to the extent that these people continue working with development projects. Otherwise, substantial amounts of money would have been spent on training them in the various aspects of development processes, only to have them return to tasks not related to rural development upon completion of the PRODAP project.

Second, the region would have a group of strengthened and, in some cases, self-sustaining farmer organizations, as well as a support system consisting of local development organizations working to provide services to PRODAP users (NGOs, municipal governments, government offices). These organizations would be able to take over much of the planning and execution of rural development in the region, working together with the Government and the Ministry of Agriculture.

Under the scenario of a new project in 2000, PRODAP would be in a better position to carry out such a project and to channel the experience gained, thus creating good prospects for compliance with project deadlines, physical and financial targets, and development objectives.

Lessons learned and recommendations

Since evaluation is a learning experience, the following comments draw on the lessons learned and could be useful in the formulation and implementation of other projects to mitigate rural poverty in El Salvador and elsewhere.

1. General

When designing projects, it is virtually impossible to predict changes that could occur in the context during the negotiation and execution periods. Accordingly, designs need to be flexible and should include contractually predetermined progress evaluations that can be used as a basis for adjusting targets, objectives and the budget by component and cost category.

Project design should rely heavily on local specialists; and, to the extent possible, those who are involved in formulating the project should also be the ones to implement it. Here it is worth recalling the limitations that have kept projects from making any substantial changes to historical situations (e.g , smallholder marketing and intermediation systems) during the projects’ time horizon.

PRODAP has shown that large project executing units can achieve good results and be well administered. However, if by the time the project closes and is transferred, appropriate arrangements have not been made to ensure the sustainability of its impact, closing can be traumatic and attention can be shifted away from sustainability as the main objective.

When designing projects, it is necessary to keep the closing and transfer phase in mind, and to indicate probable arrangements, conditions and terms of reference for the governmental and non-governmental organizations that are likely to be involved. This should be a key aspect of the mid-term evaluation.

In the future, the design stage should include an analysis of the cost differential between executing units that contract mainly with NGOs and those that are made up of officials from government agencies whose contracts include incentives for public-sector technical staff who work in project execution. Such incentives include performance contracts whereby the employee commits to specific, quantified targets in order to keep his or her job; or leasing of vehicles and equipment (e.g., computers) with an option to buy, which is advantageous for the employee and represents significant savings for the executing unit.

The success of rural development projects hinges more on the attributes of the executive director, coordinators and specialists than on the quality of the project document. Accordingly, when selecting these project officers, emphasis should be placed more on their managerial abilities than on their academic merit. This practice, which was developed by the private sector, is especially applicable to the selection process for executive directors and senior project officer positions.

Another important lesson is that a properly functioning executing unit can attract additional funds that can be channelled through project activities, thus providing greater satisfaction of the target population’s needs, including actions not considered in the original design (e.g., community works).

PRODAP, like other projects, suffered from rivalry between the so-called "production-oriented" components and the others. Such situations arise from the perception that some components mainly target group problems, while the "production-oriented" components aim at solving individual problems. In the overall analysis, however, a community should not feel better because some of its members have improved their agricultural output or acquired more land, but rather because a road has been rebuilt, a school reconditioned or a community hall built.

Projects that aim to mitigate poverty should be evaluated while they are under way. This evaluation should examine, first, the objectives that are being pursued and, second, compliance with financial and execution targets. Both quantitative and qualitative targets should be analysed, although the latter are frequently pushed into the background.

Projects intended to mitigate rural poverty conditions cannot limit their actions to the rural sphere. Since migratory flows towards the city and even abroad – and these are quite substantial in El Salvador’s case – are irreversible, projects must provide training in non-rural occupations so that migrants have better employment and income prospects.

Migration is virtually a necessity of life in El Salvador, and it seems to be closely linked to age. Since land is scarce, if it is divided among all a farmer’s children, none of them on their own has enough space to raise a family, so migration is the solution for them. By boosting production and productivity or by changing land-use patterns (except in the case of illegal crops), rural development projects cannot match the income levels earned by migrants. As a result, a broader approach to rural development is called for, one in which training is provided for farm and non-farm activities with an eye to making the migrant labour force more competitive. This would also be a way of acknowledging that the land holds a high strategic value in terms of providing a place to live and food security, i.e., a place to raise a family and educate one’s children.

The extent to which new technologies are adopted is directly linked to the cash cost involved and the time at which the cost must be covered (beginning or end of the cycle). Smallholders calculate in terms of how much they have to pay out in cash and how much they receive in cash. Food security is the first priority, with the generation of surpluses ranking second. In a scenario where the supply of land is fixed, the opportunity cost of labour is nearly zero, money is scarce and formal credit is unavailable, technologies are only adopted if they do not demand much cash and the outcomes are very stable.

The lesson to be learned here lies in the need to conduct a more thorough analysis of the processes of transfer and adoption in order to design extension systems that are more efficient and more effective. Costs must be affordable for the rural poor, so that they can be progressively assumed by smallholders and thus ensure the sustainability of the system after the disbursement period has ended.

Specific

For the Government/Ministry of Agriculture

Potential trust administrators should be identified, and the possibility of assisting the BFA with fund administration should be examined.

The advantages and disadvantages of different actions to improve portfolio recovery should be analysed, selecting the most suited ones and asking the BFA to implement them.

Specific support should be provided under the Ministry of Agriculture’s Project Coordination Office (PCO), where an evaluation unit was set up for PRODAP and PROCHALATE. The monitoring work done under PRODAP should be taken into account to redesign actions and targets. Monitoring reports could be shared with the PCO, which could supervise evaluations and/or case studies. The PCO has the necessary experience to collaborate actively in defining strategies, methodologies for the formulation and monitoring of annual plans of work, methodologies for case studies and others considered necessary for strengthening progress assessments.

Implementation of PRODAP has generated rich experience in strategies and working methodologies for development actions and activities. The managerial, administrative and technical staff working in the various components, departments and units have gained valuable experience in the execution of projects that focus on mitigating rural poverty conditions, and they have developed skills and changed attitudes. It is recommended that the possibility be considered of assigning these same people to other development projects. Otherwise, substantial amounts of money would have been spent on training them in the various aspects of development processes, only to have them return to tasks not related to rural development upon completion of the PRODAP project.

Since the characteristics of poverty vary across the project area, as do the facilities for carrying out the various components, it is necessary to identify and assign priority to specific communities in order to target effective actions that the public will quickly acknowledge and associate with results. Financial resources are and will continue to be limited, so if they are spread out over the entire area, the impact will be diluted.

For PRODAP’s Project Execution Unit (Actions Subsequent to the Interim Evaluation Mission)

The unit needs to be downsized without affecting the quality of services; operating costs need to be pared back substantially.

During its final phase, PRODAP should reassess its targets for efficiency and effectiveness, since it might be wiser to consolidate the project’s current achievements than to try to increase its coverage.

The process of selecting strategies before activities are launched is an area that needs to be strengthened. Ideas need to be exchanged prior to analysing and harmonizing the targets set in each component’s annual plan of work; this is a necessary step that cannot be skipped if the current achievements are to be built upon. Furthermore, recommending that this activity be included at the end of the project implementation period is a way of testing innovations that might make for better execution in future projects; in this connection, PRODAP should serve to verify methods that will yield higher levels of efficiency and effectiveness.

For the final phase, the Project Executing Unit should hire strategic planning advisory services to define overall project strategies as well as specific strategies for each component. These services should continue until PRODAP closes out, with two-week review and updating sessions every quarter.

In the coming months, the project should establish closer ties with the NGOs working in San Vicente and Cabañas. It should acknowledge and work with these organizations, and explore their interest in participating in the next phase of the project and ways in which they can take part in project execution (e.g., under contract with PRODAP or independently) and contribute to decision-making (transparent arrangements need to be defined to ensure real participation of beneficiaries, NGOs, government agencies, and local governments, either through an informal network of organizations or through a more structured procedure, such as a regional development foundation).

A credit analyst should be hired to deepen the various aspects of this component and recommend actions to decrease – or at least prevent any growth in – current levels of delinquency.

One of the most important lessons is that credit must be handled by specialists. Extension workers are able to plan land use and identify credit needs, but they do not want to be "loan collectors".

Land-use plans and the technical and financial feasibility analyses justifying the credit should be simplified as much as possible. It is much more important to conduct a detailed analysis of the cash flow than of total costs (including monetary and non-monetary costs), since this provides input for adjusting credit amounts and disbursement schedules. The scarcest resource for smallholders is cash, so analysis should focus on cash flows. As a rule, family labour should not be financed, since this money is normally used for consumption, it significantly raises the amount of the credit and it is very difficult to repay in the event of drought or price shocks; however, it should be financed when it is necessary to contract labour outside the family.

Functions within the Project Execution Unit should be reassigned – in line with the new strategies selected – to the extension, training, marketing and microenterprise departments, and a credit unit should be set up.

Technical assistance should also be provided to beneficiaries who are in default, provided that their delinquency is the result of problems that lie beyond their control and that they would be in a position to honour their debts if production were boosted.

The benefits of empowering communities to become active, effective requesters of government investment funds are many but they are not enough, in and of themselves, for the communities to become the conscious "owners" of a community development project. With PRODAP’s assistance, a number of communities have formulated their own proposals for improving community conditions.

This is mainly a result of the fact that communities essentially have very limited production potential. Even so, the project design, which focused on individual efforts to work the land and gave too much importance to marketing, limited families’ creativity: aside from externalizing the search for solutions, it reduced the scope of the problem to obtaining credit for boosting production and being concerned with prevailing price levels at the time of sale.

PRODAP lacked a component to organize the project’s beneficiaries for purposes of receiving specific project actions. Although this was not practical at the beginning due to conditions imposed by the project’s formulation (as discussed above), in the course of project execution and in light of the evolving context, a department should have been set up within the Project Executing Unit to oversee organization of the project’s beneficiaries and coordination of the various project activities within each geographic region. The overarching objective of the department would be to ensure "ownership" by the target population and thus guarantee sustainability during the post-loan period.

For IFAD

To ensure a reasonable level of sustainability for PRODAP’s impacts, it is recommended that the project not close until the beginning of the year 2000, provided the Project Executing Unit undertakes to implement the remedial actions described herein. Otherwise, the proposed date will arrive without there being any better chances of sustainability of the project’s impact than there are at present.

When formulating new projects, designs should not only be flexible but they should also include a contractual obligation to conduct a mid-term evaluation. This will provide feedback on adjustments that need to be made in project components and activities as well as in the allocation of resources by category.

The design should clearly show whether the project is a rural or agricultural development project or a credit project. Judging from PRODAP’s objectives, it would appear that this is a rural development project; looking at the amount earmarked for credit purposes and considering the emphasis of the project’s actions and the focus of the supervision activities, it seems to be a credit project.

The Interim Evaluation Mission found that the credit component was clearly oversized in the original design and that the situation was not addressed subsequent to the baseline study and its findings. This excess supply of credit led to forced demand, i.e., extension workers granted credit with low probability of recovery, as shown by the mounting levels of delinquency and the current high-risk situation.

Project designs should set the last year of a project aside for consolidating the project’s achievements. It is not wise to take on new beneficiaries at this time, since the quantity and quality of actions are affected strongly by the closing phase. As part of the actions to consolidate and ensure the sustainability of a project’s impact, the closing and transfer stage should be planned for and described in the formulation document.

PRODAP used considerable resources in its marketing efforts, although these had little impact on the project’s overall results. The emphasis should not be on creating new marketing channels but rather on improving and/or expanding channels that were already in place before the project. Farmers normally do not stay with new channels for long, mainly because of their lack of confidence in the sustainability of the new, "imposed" channel, the risk of losing their connection with traditional marketers if the project fails, and the lack of arrangements for cash advances during the crop cycle.

An effort should be made to avoid excessive optimism or creating expectations with regard to marketing. The risks and costs of any proposed replacement process need to be gauged, avoiding simplistic analyses that might show that the organizations created under the project could easily replace intermediaries that are specialized in marketing; it should not be forgotten that the projects are temporary while the intermediaries have been and will continue to be around for a much longer time.

Nor should it be forgotten that a project’s objective is not to make farmers into marketers or merchants, but rather into better farmers, with lower unit costs and better quality product thanks to a market vision (adjusting supply to demand) that makes them more competitive as a result of better quality. Thus far, achievements in terms of organizing a feasible marketing system for marketable surpluses have been very rare, very limited in nature and not sustainable.

Marketing actions need to take an entirely different, innovative approach, so as to break the chain of discouraging experiences in terms of achievements, effects and impacts.

Projects should include training for intermediaries, since these organizations generally operate on the basis of practical knowledge that they have accumulated over the years.

 

Jump to Project design and objectives Target group Objectives and components Expected effects and assumptions Evaluation Implementation context Project Achievements Effects assessment and sustainability Lessons learned Read more IFAD operations in El Salvador Hot links Project evaluations