Interim Evaluation

An innovative and complex project

The Rural Development Project for Ngöbe Communities was an innovative project in that it included an ethnic rights objective-legal recognition of the Ngöbe-Buglé Comarca-which endowed the project with very special political characteristics stemming from the involvement of indigenous leaders in the Steering Committee and the provision of training to prepare those leaders to carry out planning and administration for the Comarca. The lasting impact of this objective and the legal recognition of the Comarca in 1997 confirm that the design was correct from the strategic standpoint. Moreover, the training provided for Ngöbe-Buglé technical personnel, community leaders, and producers involved in the project proved to be a positive investment in human capital for the Comarca that will undoubtedly continue yielding results in the future.

The project also included two components that posed a significant technical challenge: the improvement of family incomes based on agricultural production in an agriculture-ecological environment with restrictions and the establishment of the Capitalization Fund to finance productive activities, which was to be administered by grass-roots beneficiary organizations that lacked an organizational tradition and had weak administrative and managerial capacity. The limited results achieved under both components confirm that during project execution more emphasis should have been placed on analysis and adjustment of these two lines of action in order to enhance their effects and impacts. Notwithstanding these limitations, the project did result in improved family food self-sufficiency, thanks to the introduction of new productive activities.

Project design

The design of the project, which drew on the lessons learned from the Rural Development Project for the Guaymí Communities, was on target in several respects, for example: inclusion of an intensive training programme as a basis for organization of the beneficiaries, promotion of their participation in planning and management of the project, creation of a Project Executing Unit (PEU) to coordinate activities, contracting with the United Nations Development Programme (UNDP) to serve as a project administrator, construction of development centres to decentralize the development activities, establishment of realistic guidelines for the formulation of an agricultural technology proposal that was simple and appropriate to the potential of the area, incorporation of studies on the status of natural resources and the impacts of productive activities, and environmental protection based on the promotion of an environmentalist attitude among the population.

Nevertheless, the project design missed the mark in other areas, such as entrusting the majority of the activities to public-sector institutions without establishing effective mechanisms for coordination with the PEU; miscalculating the implementation period for the Capitalization Fund; overestimating the willingness of groups of farmers to pay directly for technical assistance; not planning for marketing support services; misjudging the extent to which the Ministry of Public Works (MOP) would be able to improve the road system in the area, which was only partially accomplished; and in the incomplete organization of the Administrative Unit of the PEU.

Overview of project execution

The project was carried out in a favourable macroeconomic, political, and climatic context, characterized by moderate but sustained growth of the national economy. In addition, monetary and price stability prevailed, as a result of several factors: implementation of economic policies aimed at enhancing the efficiency of the state; the government's acknowledgement of the unfair distribution of wealth among the Panamanian population and its especially detrimental effect on the indigenous populations; the formulation of social policies targeting the poorest groups; legal recognition of the Ngöbe-Buglé Comarca, an achievement which the project was instrumental in bringing about; stability of domestic agricultural prices, coupled with an increase in the price of coffee and beans; and the absence of climatic disasters, except for some negative effects of the El Niño phenomenon in 1997.

However, the fact that there were three changes of government during project execution led to the replacement of the authorities within the executing agency and the representatives of the other public institutions involved on the Steering Committee. Moreover, the representatives of the Ngöbe-Buglé General and Regional Congresses changed several times. These changes translated into a high degree of instability in the staff of the PEU, which had 5 executive directors, each of whom served for an average of only 15 months, in addition to two officials who served as directors on a temporary basis. This turnover also affected other key positions within the PEU; the 27 officials in charge of project components and administration remained in their posts for less than 14 months, on average. In sum, the 8 key positions within the PEU were held by 32 different people, who served for an average of 14 months each, although it should be pointed out that 8 of those 32 people were promoted from lower positions within the PEU.

The foregoing difficulties delayed start-up of the project, which was launched in 1994, but some components did not get under way until 1997. It is worth noting that 1997 was also the year that Ngöbe-Buglé professionals joined the executive management of the PEU. The high turnover of administrators within the PEU, coupled with the shortage of administrative staff and the lack of importance attached to administrative and accounting procedures, resulted in an appreciable delay in the disbursement of funds for IFAD loan 331-PM-so much so that the amount authorized for the Special Account was disbursed in 1995, but only in 1997 did the first replenishment of funds take place. Despite these problems, however, thanks to timely action by the Social Emergency Fund (FES)/Social Investment Fund (FIS), the entire counterpart contribution was paid on time, which made it possible to maintain project operations when the IFAD loan funds could not be disbursed.

Generally speaking, the organization of the PEU was adequate, but the bulk of resources were allocated to the central office, to the detriment of the development centres, which lacked sufficient personnel and office and transport equipment to carry out the functions assigned to them according to the project design and execution.

A noteworthy aspect of the project was the stability and the regularity of sessions of the Steering Committee, which met 55 times over 7 years and dealt with all important project-related matters.

However, the overall performance of the project depended more on the performance of the public institutions that collaborated in its execution than on the functional organization of the PEU, and the failure of some of those institutions to fulfil their commitments limited the project's sphere of action. As a result of these failings on the part of the co-executing institutions, many activities were not carried out or were left to the PEU, whose technical staff was overburdened with responsibilities. The deficient performance of the public-sector institutions was due both to their own operational limitations and to the reluctance of the project to provide them with equipment and resources for operations, even though it was to have done so under the agreements established.

Another positive aspect of the project was the mechanisms for beneficiary participation, which were built into the project design. Those mechanisms were complemented by new forms of participation, as a result of the evolution of the beneficiary organizations. The basic mechanisms for participation were: (i) formulation, monitoring, and evaluation of the annual work plans of the development centres by the producers councils, composed of leaders of grass-roots groups and community organizations; (ii) inclusion of four Ngöbe-Buglé leaders on the Steering Committee, the supreme decision-making body for the project; and (iii) participation of Ngöbe-Buglé representatives on the Regional Credit Committees.

One obstacle that the project had to get around was the reduction of the financing by USD 2.1 million (15%), which was to be contributed by the World Food Programme (WFP) and the United Nations Development Programme (UNDP). The impact of this loss of funding became apparent only after the start of the year 2000, owing to the project's low initial level of financial execution. Nevertheless, even prior to that year there were some budgetary constraints for the Production Development and Training/Organization components. The project also received complementary financing from the Japanese International Cooperation Agency (JICA), which was used to supply agroindustrial equipment and technical assistance to eight of the cooperatives served. It should be noted that the Government of Panama contributed the entirety of the counterpart resources committed and that between 1994 and 1996 the project operated mainly with national funds.

Principal activities and their effects and impacts

One of the project's most important accomplishments is its contribution to the recognition of the Ngöbe-Buglé Comarca, since the achievement of that legal status will ensure effective protection of the territories of the Ngöbe-Buglé people, establishment of their own forms of government, and maintenance of their way of life and traditions. The most noteworthy activities in this area were the economic and logistic support provided to the Intergovernmental Commission of the Ministry of the Interior and Justice and to the indigenous leadership, the dissemination of laws 10 and 69 and Charter of the Ngöbe-Buglé Comarca, and the logistic and financial support for the various General and Regional Congresses held during the project execution period.

Also significant, both in terms of their coverage and their impact, were the training activities, which sought to strengthen the social organization of the beneficiaries, develop participatory assessment and planning, promote application of the gender perspective, enhance institutional and administrative organization of the Ngöbe-Buglé Comarca, improve agricultural production, and raise awareness of environmental issues and promote widespread adoption of conservationist farming practices.

The training imparted under the Training/Organization component was targeted at four different types of participants: producers in the grass-roots groups, leaders of the cooperatives and associations, technical personnel of the PEU and the institutions involved in co-execution of the project, and indigenous leaders. Some 1 800 events were carried out, the majority (93%) for producers, while 5% were for technical personnel PEU and the co-executing institutions, and the remaining 2% were for indigenous leaders. The participants in all these events totalled 10 254 producers (86%), 1 176 technical staff (10%), and 869 Ngöbe-Buglé leaders (4%). Women's participation in events of the UOC reached 33%. However, the records on participants, which do not identify them by name, make it impossible to establish the exact number of people trained, since some people attended more than one event.

Thanks to the training and organization-building activities, there are currently 20 community organizations, almost all of them legally established, including 14 cooperatives and 4 producer associations with a total membership of close to 1 000. In addition, this component of the project served 336 grass-roots groups composed of farmers, artisans, environmentalist committees, and pro-works committees. It should be pointed out that several groups were never consolidated and almost 40% later broke up. Nevertheless, the beneficiaries interviewed by the Mission expressed satisfaction with this achievement, since the Ngöbe-Buglé cultural tradition emphasizes family work (not group work), and they had also suffered bad experiences previously. Another form of organization was producer councils formed within the development centres and composed of representatives of the groups associated with each centre. The councils began to form in 1998 as an initiative of the producers to evaluate fulfilment of the annual work plans and monthly work programmes of each centre, and they have become an important mechanism for beneficiary participation in the project, with the potential for ongoing self-management.

The training offered on the gender perspective also yielded extremely positive results, as evidenced by women's involvement in the majority of project activities. For some of those interviewed, this signified a positive break with tradition, since Ngöbe-Buglé culture assigns women a secondary position in the social milieu. The signs of this cultural change can be seen in the participation of women in the beneficiary organizations: 309 women participated as members of community organizations and 47 occupied leadership positions. The grass-roots groups had 1 626 female members, with 494 in leadership positions. This enhancement in the social status of women also extended to the level of the Comarca, as shown by a Women's Congress in which more than 2 000 women took part. Moreover, until March 2001, a woman served as Vice President of the Ngöbe-Buglé General Congress, and at the last General Congress (March 2001), one of the three presidential candidates was also a woman. Within the PEU, women consistently made up 30% of the staff , and professional Ngöbe-Buglé women had risen to several managerial positions by the end of the project, including those of Executive Director, Coordinator of the UOC, Director of the Monitoring and Evaluation Unit (MEU), and Coordinator of the Alto de Jesús Development Centre.

The environmental training was aimed at fostering the protection of natural resources and also disseminating the legal framework relating to the environment. The participants in the training numbered more than 2 000 and included technical personnel from the PEU, Ngöbe-Buglé leaders, and producers. The topics covered included environmental legislation, soil and water conservation, environmental conservation (flora and fauna), and agroforestry system management.

Almost all the training mentioned above was imparted through brief events without any organized curriculum or further training events, except for the Sustainable Agroforestry School, which was a training programme in organic and conservationist agriculture. The latter consisted of eight modules of one week's duration, offered over eight months and aimed at producers selected on the basis of the high probability of their remaining in the community and their willingness to transfer their knowledge to other community members. Each module covered a specific topic in organic farming-such as soil conservation techniques, biological pest control, development of organic fertilizers, and others-which included both theoretical and practical components and required the participants' to devote themselves full-time to their studies. At the end of each module, the participants returned to their property and managed a plot of land applying the knowledge they had acquired. The evaluations took into account not only participation in the courses, but also application of the knowledge acquired when working the land. Those who completed the programme successfully received a diploma certifying them as promoters. A total of 326 producers entered the school between 1998 and 2000, with 210 (64%) graduating as promoters.

The activities in the area of production development included training, technical assistance, and technology assessment. Assistance was provided with regard to traditional crops (corn, rice, beans, cassava, and coffee), new productive activities (flood rice farms, fish farming, rice-fish farming, and integrated farms), and complementary activities, targeting groups of women in particular (poultry and pig breeding). Some non-agricultural microenterprises were also promoted (manufacture of traditional clothing, crafts, bakeries, and restaurants), though to a limited extent.

The technical assistance for farming served 3 018 families belonging to 507 producer groups. As part of the technical assistance activities, technical experts from the Ministry of Agriculture (MIDA) and the PEU created demonstration plots for various crops. However, this technical assistance was discontinuous and in general did not occur concomitantly or as follow-up to the training provided, owing to a shortage of technical staff within the PEU and the co-executing institutions.

The training in productive aspects under the Production Development component was provided by MIDA technical experts and, occasionally, by technical staff from the PEU. More than 1 500 short-term training sessions were offered on various topics. Over 57 300 people (64% men and 36% women) participated, including not only producers who were part of the groups associated with the project, but also other members of the communities served. As with the Training/Organization component, the records on participants did not identify them by name, so the exact number of people trained is not known, since several people attended more than one event.

The results of these activities were access to technical assistance and training for indigenous families, most of whom had not previously received services from MIDA. Many families also launched new productive activities that improved the availability of foods for family consumption; these included integrated farms-66 farms for 241 family groups-and complementary activities, such as free distribution of 473 pig and poultry modules, the construction of 33 ponds for rice-fish farming (500 beneficiaries), and the construction of 106 ponds for fish farming (1 162 beneficiaries). In addition, new production technologies were introduced, such as soil conservation techniques (107 ha), use of organic fertilizers produced on the farm, simple irrigation techniques for flood rice production (9 ha), and coffee management techniques. Several technologies focused on minimizing the use of commercial inputs, thereby avoiding the need to purchase them. Others helped improve the quality of seeds for basic grain crops (rice, corn, beans) through the introduction of some commercial varieties previously unknown to the indigenous population (212 seed banks).

These results were encouraging, but they were limited in scope because the work carried out under the component was affected by the limitations of the co-executing institutions (e.g., only 10 MIDA extension agents, who lacked sufficient means of transportation). Further, the MIDA technical personnel did not have enough experience in group extension methodologies, and the researchers from the Agricultural Research Institute of Panama (IDIAP) had little experience with the prevailing production technologies and conditions among the Ngöbe-Buglé population. In addition to the aforementioned problems, there were other limitations, including the following:

1. Extension methodology problems stemming from the delayed and incomplete implementation of the Capitalization Fund, which prevented the hiring of technical personnel by the beneficiaries;
2. Suspension of the WFP financing, which reduced the possibility for disseminating some technologies, especially the implementation of soil conservation works;
3. Centralization of resources within the PEU, to the detriment of the development centres;
4. Insufficient coordination of activities between components and even within the Production Development component, since new productive activities were introduced without providing the beneficiaries with adequate training;
5. Paternalism in the technical assistance, since not only were inputs, materials, and tools provided free of charge, but the technical experts took responsibility for purchasing them;
6. Technical assistance that overemphasised productive and technological aspects, with insufficient attention to marketing and management;
7. Training limited to training events, with no follow-up on the application of technologies through the provision of technical assistance to the grass-roots groups;

Serious problems occurred in the implementation of the Capitalization Fund sub-component because the first set of credit regulations, approved in February 1996, limited the creation of revolving funds owing to the characteristics of the cooperatives and associations. There were also administrative problems in the disbursement of the approved financing. The original regulations were modified to enable grass-roots groups to receive project funds directly, but they provided that the recovery of funds would remain the responsibility of the cooperatives and associations, which would continue to manage the revolving funds. This modification was intended to stimulate fund disbursements and increase the number of credit recipients; however, the training provided to the groups on credit management delayed the approval and disbursement of the funds. In addition, to expedite the approval of loans, in 1999 three regional credit committees were established, but approval and disbursement procedures remained slow because seven different entities were involved: the Capitalization Fund Unit of the PEU, the Steering Committee or the Regional Credit Committees, Social Investment Fund (FIS), the Office of the Comptroller General of the Republic, the Ministry of Economy and Finance (MEF), the Administrative Unit of the PEU, and the suppliers of the inputs acquired.

The initial financing was disbursed in 1997. The amount approved as of 31 December 2000 was 728 000 balboas, distributed among 10 cooperatives, 1 producer association, and 258 grass-roots groups. Nevertheless, because of the lengthy procedures for authorization and disbursement of the funds, as of that date only 341 000 balboas had been disbursed to 4 cooperatives, 1 producer association, and 124 grass-roots groups, benefiting 1 518 indigenous families.

The Capitalization Fund resources were devoted to agricultural activities (56%), especially processing and marketing of coffee, followed, in order of importance, by commercial activities (37%, including stores selling consumer merchandise), livestock activities (6%) carried out by 61 grass-roots groups composed primarily of women (poultry farms), and craft activities (1%) by six groups made up of 32 women.

The results yielded by the Capitalization Fund can be considered limited, since they assisted only 30% of the expected target population, although the targeting was adequate, given that the beneficiaries (low-income groups and women) lack access to any other formal financial services. The development effects cannot yet be established because, owing to the short duration of the programme and the fact that the beneficiaries received a single loan, no obvious positive results- such as increased income and/or creation of new jobs-can be discerned. Moreover, the Fund's institutional and financial sustainability is doubtful because the interest rates were not sufficient to cover the costs of administration, currency devaluation, potential default, and capitalization. Furthermore, accounting practices for the project and the community organizations were deficient in that they did not allow for good monitoring of the Fund.

The environmental protection activities consisted of training for technicians and beneficiaries; specialized training for student leaders from the Comarca through the awarding of three fellowships for the study of environmental issues; sponsorship for a graduate thesis on a specific environmental topic; promotion of the formation of several environmentalist groups, three of which have already established by-laws and are in the process of gaining legal recognition by MIDA; and three environmental studies-one on the status of natural resources in the area, compilation of information on natural resources as part of the baseline study (not yet complete), and an environmental impact study of the Kenani Lagoon channel. However, the technical studies called for in the project design were not carried out.

The environmental training yielded positive outcomes in that it helped foster a pro-environmental attitude favouring the protection of natural resources among the population. This attitude was evident in the interviews conducted by the Mission and is evidenced by a significant decline in hunting, fishing, and indiscriminate felling of trees. Hunting is now permitted only for subsistence, and trees may be cut only for the construction of housing. Progress was also made on the protection of waters sources in Cerro Iglesias through the issuance of a declaration on preserving forest cover to protect water sources, Agreement 001-99 of the National Environmental Authority (ANAM), which benefits six communities that are home to some 200 families. Still pending is the establishment of the Administrative Board of the Water Supply System, which will be legally responsible for enforcing the Agreement.

Another activity that yielded good results was the installation of 822 latrines (serving 820 families and 2 schools), with support from the Ministry of Health and the Water Project of the United Nations Children's Fund (UNICEF). This activity grew out of the regional participatory assessments, which identified latrine-building as a high priority for the communities in order to reduce contamination of surface and ground water and prevent the spread of water-borne diseases.

As for the forestry development programme, its outcomes were the training of beneficiaries and technical personnel in methods for reforestation in the area and creation and management of test plots; 93 tree nurseries were established-65 for community plantations and 28 for individual plantations-benefiting close to a thousand families; and 105 ha that protect six micro-watersheds were reforested. Although these results were technically encouraging, they fell short in terms of coverage, based on the expectations set out in the design.

The activities carried out under the infrastructure component were construction of 11 development centres-one more than expected; improvement of 83 km of rural roads, and arranging with other institutions for an additional 54 km of road improvement; construction of 7 suspension bridges for use by pedestrians and beasts of burden; improvement of 23 walking paths (93.5 km), for which the communities contributed the labour; and execution of two road studies, one for a road between Cerro Sombrero and Llano Tugrí, future seat of government for the Ngöbe-Buglé Comarca.

The road improvements began in 1996, when a contractor company was retained, and activities intensified in 1998, with the supply of road-building machinery purchased through international competitive bidding. The goal established in the design (improvement of 30 km) was surpassed, but the technical specifications for the works were not followed (compacting of the road surface, resurfacing of the steepest roadside drainage ditches), and the improved roads therefore deteriorated quickly. In addition, the improved roads were not maintained, which contributed to their rapid deterioration. During 1998 and 1999, the MOP operated the machinery and provided the agreed contributions-Regional Veraguas-but subsequently project personnel had to take over operation of the road machinery because the contributions from the MOP ceased. Overall, the effects of the roads were positive, since the availability of transportation increased and there was a partial reduction of fares.

The development centres had a clear positive impact in that they helped initiate the decentralization of project activities and also served as a gathering place for beneficiaries, Ngöbe-Buglé leaders, and technical personnel from the PEU and the co-executing institutions. The quality of the construction was good, and improvements were introduced into the design, increasing the area covered. The works began in 1995 and were completed in 2000, which was slightly longer than planned. Maintenance has been generally good, although some repairs are needed and additional maintenance work is required. However, the equipment is insufficient, both in comparison with what was envisaged in the design and in terms of current needs.

The suspension bridges were well built, but only 7 of the 20 bridges planned were actually constructed. Contracts for an additional three bridges have since been awarded to contractors and the formalities are being completed. With regard to the improvement of walking paths, the established goal was achieved, and labour for works was provided by the beneficiary communities, as were the advisory services and topographic surveys by the MOP. All the works constructed are being maintained by the pro-works committees to the extent of their capabilities. Women are active participants on these committees, both in the organization of fund-raising events and the performance of maintenance work.

The activities carried out under the Monitoring and Evaluation component were annual operational planning, monitoring of annual work plans and monthly work programmes, and evaluation. The project design called for the evaluation to be carried out in thirds, as a result of which the Monitoring and Evaluation Unit (MEU) focused on planning and monitoring, postponing the evaluation somewhat. In 1998, woman responsible for Gender Issues within the PEU was brought into the Unit with the objective of ensuring application of the gender perspective across all aspects of the project. The MEU also provided training to facilitate the participation of beneficiaries in planning, monitoring, and evaluation of project.

The baseline study, which was to be used for the strategic planning of the project, was not carried out at the scheduled time. The study was contracted for in 1999, but the work was not completed and the final report is not available. The design of the monitoring and evaluation system was not prioritized, either. The system was not was developed until late 1997, and it was only partially implemented.

The annual planning, which initially was carried out exclusively by the PEU, gradually involved the beneficiaries, first through meetings in the field with indigenous leaders and the technical personnel from the co-executing institutions, later in accordance with the plans drawn up by the communities, and finally taking into account the demands identified in the participatory rural assessments, which were prioritized by the producer councils in the Development Centres.

Monitoring was based on the compilation of information on fulfilment of the monthly work programmes (field records) and the submission of periodic reports by Development Centres and of the various units within the PEU. This information was used as input for the meetings of the Steering Committee, the PEU Coordinating committee, and the annual workshops on participatory overall evaluation. Evaluation was promoted and coordinated directly by the MEU-no outside personnel were hired-through the mechanism of the annual evaluation workshops, which gathered information on some indicators. Nevertheless, the project did not have a true system of monitoring and evaluation because the evaluation and monitoring activities were not linked; rather, they were carried out in an isolated and not very systematic manner and did not utilize the project objectives and community plans as a basis.

Project supervision was entrusted to two cooperating institutions, the United Nations Department of Economic and Social Development (UNDESD) until 1997 and subsequently to the Office of Rural Development Operations of the Andean Development Corporation. The two agencies carried out a total of 13 missions at regular intervals. Both also administered IFAD loan 331-PM. This supervision was complemented by nine missions carried out by the Latin America and Caribbean Division of IFAD, which addressed specific issues and special situations that required the presence of the Fund. In the Mission's view, the intensity of the supervision and the issues addressed were appropriate, although some weaknesses in the project were not rectified.

The project received technical assistance from several entities: the Latin America and Caribbean Division of IFAD, the Office of Evaluation and Studies of IFAD, the Regional Programmes of IFAD, the cooperating institutions, the Panama Office of the UNDP, and the consultants hired by the PEU. Technical assistance was provided for 19 missions overseen by 13 experts, which dealt with the most complex matters, such as monitoring and evaluation, administration and accounting, Capitalization Fund, training, gender issues, organization and methods and functionality of the development centres. In addition, technical personnel from the PEU participated in several workshops of the IFAD Regional Programmes.

Specific recommendations

These recommendations are aimed at consolidating the project's achievements. Some can be taken on and executed by the project, though others will require the participation of the governmental institutions involved in project execution, whose mandate is to assist in the development of the Ngöbe-Buglé Comarca.

Monitoring and evaluation: The information available in the project monitoring and evaluation system should continue to be systematized. It is also important apply the methodology used in the baseline survey for the final evaluation, as doing so will yield more consistent and reliable information for determining the effects and impacts of the project. In addition, bearing in mind that a plan for development of the Comarca is being prepared, it is important to link the community development plans and the annual work plan of the project with the plan for the Comarca, which will make it possible, in the final evaluation of the project, to assess not only the degree to which its objectives were achieved, but also its concrete contribution to the development of the Comarca.

Training: With regard to building local capacity for development, an evaluation of the training system is needed to provide a basis for programming training activities under the new project. That evaluation should analyse the strengths and weakness of the communities and the beneficiary organizations in terms of the achievement of the long-term objectives of the community development plans and the plan for development of the Comarca. In order to reinforce the learning achieved, it is important also establish mechanisms for ensuring in-service training and follow-up with the producer councils and the producers involved.

Production development: In order to assure the sustainability of the effects and impacts achieved and contribute to the creation of a better extension system in the future, the following actions are recommended:

1. Transfer, to the extent possible, materials, and means of transport to the development centres, which could be managed by the producer councils;
2. Consolidate the productive activities fostered by the project, reinforcing the training on conservationist technologies and marketing processes;
3. Ensure the participation of the promoters trained at the Sustainable Agroforestry Schools in a future IFAD project in the Ngöbe-Buglé area;
4. Promote the modality of volunteer technical assistance used in the JICA programme.

Capitalization Fund: The activities planned under this subcomponent were not completed, and new credit regulations are still being drawn up and preparations are ongoing for the establishment of a second-tier organization of cooperatives to manage the revolving funds. In addition, close to USD 500 000 approved by the Steering Committee and approximately USD 550 000 in available fresh funding remained undisbursed. The mission believes that this second-tier organization is not very feasible in the short term. Moreover, the community organizations have little experience and need to be strengthened with an eye towards their consolidation. It is therefore recommended that a nongovernmental organization with credit experience in the area be retained to manage the funds, which would be accessible to community organizations in the Ngöbe-Buglé Comarca, both for their own productive activities and for the extension of loans to members or grass-roots groups. It is important that the latter not have direct access to the Fund and that the conditions for sub-loans, including interest rates, be established by the community organizations.

Future use of the road machinery: The machinery was acquired to improve and maintain the roads of the Comarca with the ongoing participation of the MOP. However, this did not fully occur, which led to problems that affected the quality and stability of the works. This situation makes it essential to consider new ways of utilizing and operating the machinery in order to ensure that it remains in the Comarca and is operated efficiently.

Environment and reforestation: The environmental protection and forest development activities should be part of a plan for environmental management of the Comarca. It is therefore recommended that the ANAM develop such a plan, in fulfilment of its mandate. The plan should also identify and appraise the productive activities compatible with rational use of natural resources with a view to restoring and preserving the ecological balance in the Comarca.

Administration: The current accounting records are inadequate to complete the administrative closing of the project. Hence, it is recommended that accounting practices be normalized during 2001 so that the project's assets can be transferred, as appropriate, to beneficiary organizations. This transfer of assets should be based on an updated socio-economic assessment of the applicant organizations.

Budgetary and financial issues: The budget approved for fiscal year 2001 (USD 1 000 000) is clearly insufficient to complete the execution of the project activities under way, which include preparation and updating of the community development plans, implementation of a system for ensuring the sustainability of the Capitalization Fund and transfer of the development centres (equipped and in good repair), in addition to ensuring that the road machinery remains in the Comarca. Accordingly, it is recommended that the FIS and the MEF take the necessary steps to obtain a budgetary increase as soon as possible in order to conclude the activities during 2001. IFAD might also consider modifying the pari-passu for the remaining loan funds.

Conclusions, general recommendations, and lessons learned from the experience

The efforts of the FES/FIS, MIPPE/MEF, IFAD, and the cooperating institutions aimed at overcoming the technical, organization, and functional limitations of the project were unsuccessful in many cases. This was mainly due to the frequent changes in representatives to the Steering Committee and high staff turnover within the PEU.

Participation by traditional authorities and representatives of the political-administrative entities of indigenous communities in the management of a project aimed at a homogeneous ethnic group is an essential condition for execution of the project and for participation of the beneficiaries.

It is important to formulate long-term community development plans to serve as a guide for the preparation of annual community plans, which, in turn, serve as a basis for the annual work plans for the project. The community development plans and the plan for the project should also serve as input for participatory evaluation processes. These measures will ensure that the activities are coherent and based on a long-term vision, which will lead to greater possibilities for the sustainability of the processes generated by projects.

Agreements with co-executing institutions should include, at a minimum, the activities to be carried out, the specific goals and expected outcomes (effects and impacts), and also the indicators that will be used to verify the achievement of the outcomes. These agreements should be revised and adjusted annually.

The organization and implementation of project administration should be appropriate and sufficient-in terms of the annual expenditure level-for the administrative activities required. The design should specify the administrative and accounting procedures to be used and the staffing needs for the project and should also identify the computer capabilities required to administer several sources of financing.

In demand-oriented projects, support for existing organizations-before creating new organizations-facilitates organizational strengthening of the target population and increase the possibilities for sustainability. In addition, the emergence of relevant local initiatives should be supported because they will help empower the beneficiaries and enhance the sustainability of local development processes. In this connection, the application of a cross-cutting approach to training in a demand-oriented development project will not be very effective if it is not articulated with the long-term objectives of the community development plan and the strategic plan for the project.

Application of the gender perspective across all aspects of the project by the technical team of the PEU increases the likelihood of achieving positive results in the communities and organizations served.

The approach of "cascade training" should be accompanied by concrete mechanisms to provide in-service training for those who will be expected to replicate the knowledge and abilities they have learned in order to ensure the quality of the training received by people at the lowest levels of the "cascade." This type of approach is not very effective in terms of generating significant local capacity if it is limited to training of a single representative or delegate per organization or if it focuses strictly on training managers to perform their functions, since it will not assure the continuity of processes if the managers change.

The training for production development and the extent of its impact yielded the following lessons: (i) short-term training with broad coverage is important for disseminating new technologies, but greater relative emphasis should be placed on more intensive training with fewer participants; (ii) training for technicians and producers, especially the leaders of organizations devoted to the marketing of products, should incorporate matters relating to markets and marketing; (iii) although it is logical for several components to include training activities, the central role of the training component should be clearly established with regard to the prioritization of contents, types of beneficiaries to be served, and coordination of the training with the rest of the activities; and (iv) it is essential to control the quality of the training as part of the monitoring activities.

Public extension institutions may not possess the capacity to provide appropriate services to indigenous populations, which requires a knowledge of the predominant production systems, a suitable technological-productive proposal, and extension methodologies that take account of the characteristics of the beneficiary population (e.g., language, high level of illiteracy). It is therefore desirable to consider employing alternative systems that utilize community members who have received intensive training and who live in the communities and are able to communicate in their own language. This means establishing an appropriate training programme and mechanisms for providing refresher courses for the promoters trained.

Establishing and administering a credit fund is a complex undertaking that requires specialized skills, for the following reasons: (i) application of a financial policy for a project is difficult because various interests exist which may be at odds with financial management standards; for this reason, the role of the project should be mainly to formulate policies and then contract for administrative services from a specialized financial entity; (ii) as specified in the design, the loans were made in kind in order to control investments by the beneficiaries; the experience of other projects with regard to this modality of disbursement has not been positive because the borrower's management capacity is underestimated and also because direct responsibility is assumed in the financed activity, which increases the transaction costs (transport, storage, losses, and others; and (iii) no provision was made for a savings component, although experience with microfinance has shown that an intermediation system is not sustainable in the long term without mobilization of the users' own resources; furthermore, those savings may serve as an element for client selection and may be used as collateral.

With regard to the Physical Infrastructure component: (i) the effective participation of the communities, through the pro-works committees, in the selection of the works to be carried out and in their later maintenance contributed significantly to the positive effects achieved; and (ii) when co-execution modalities with public institutions are established, the project should have guarantees of the technical capacity and the availability of resources from the co-executing institution; in addition, the commitments should be clearly spelled out in the agreements, as should mechanisms for evaluation, revision, and adjustment.

The Environmental Protection component also yielded the following lessons which could be applied in the design of similar projects: (i) lack of information on the environmental situation at the outset limits the ability to assess the effects and impacts of the project and hinders the development of indicators to guide decision-making during project execution; and (ii) lack of medium- and long-term plans impedes the continuity of activities, especially when there is a change in personnel. This planning, which should be updated periodically, should be the starting point for the annual work plans developed jointly with the beneficiaries and the environmental authority.