Interim Evaluation
Exectuive Summary

A. Background

1. In April 1996 the President of the International Fund for Agricultural Development (IFAD) recommended that the Executive Board approve a new technical-assistance grant for the Group for Agricultural Research (GIA) of Chile, for the purpose of carrying out the Regional Training Programme in Rural Development (PROCASUR). The programme was launched in August 1996 with a view to building the management capacity of IFAD projects in the region and fostering a market for training services (IFAD’s regional programmes are referred to as "programmes", while national projects are "projects". As used herein, the term "programme" refers to PROCASUR). The programme’s objective was to create an organization that would assume a commitment to continue the training actions. To achieve this, the agreement between IFAD and GIA stipulated that the following activities were to be carried out: (i) technical assistance on methodologies; (ii) training for technical staff, farmers and members of indigenous groups; (iii) support for the regional market for training services, databases, publications and dissemination; and (iv) monitoring and evaluation.

2. For operational aspects, IFAD recognizes GIA as the executing agency responsible for administering funds and implementing the respective investment-related decisions. GIA set up a Regional Coordination Unit (URC) and an Advisory Council made up of five regional agencies selected by the URC. A Regional Training Support Fund (FDRC) was to be set up and would be transferred to the new institution as of the third year.
3. The expected contribution of USD 150 000 from the Andean Development Corporation (CAF) did not materialize, despite the fact that PROCASUR had been identified as a programme that would enhance the efficiency of country projects for which the CAF was a cooperating institution. The contributions expected from the European Union, which had supported the first and second phases of the programme, also did not materialize. The programme’s resources came to a total of USD 2 730 000.
4. As at the end of June 1999, contributions to the FDRC totaled USD 430 535 and had been donated mostly by IFAD (USD 350 000) and the programme’s partners (the Regional Programme to Support Indigenous Peoples in the Amazon [PRAIA], the Regional Programme for the Development of South-American Camelids [PRORECA], Monitoring, Analysis and Evaluation for Development [SASE], the Centres for Group Studies and Projects [CEPA], Appropriate Technology International [ATI], the Farm Marketing Development Institute [IFOCC], GIA, and the University and Region Research Institute [IIUR]). The working capital needed by PROCASUR to finance the central unit and all the training activities was provided mainly by the clients. The resources used by the programme exceeded USD 2.7 million, although neither the European Union nor the CAF followed through on their initial commitments.

B. The Setting

5. The geographical areas in which IFAD projects are under way display a wide variety of social, organizational, economic and environmental conditions, which creates special needs in terms of project design and implementation. This in turn holds special importance for the achievement of PROCASUR’s objectives. Special mention should be made of the adverse climatic conditions, degradation of natural resources, and the precarious nature of local business, labor, and municipal organizations. The building of stronger organizations was identified as one of the key factors that would make it possible to carry out projects to resolve structural problems and create conditions to alleviate poverty. In many of these areas, it is difficult to overcome the conditions of poverty through smallholding-based agriculture. Other options, such as non-farm investments, have yet to be explored fully, since (aside from a few exceptions) the focus continues to be on agriculture rather than rural development. If fresh alternatives are to be explored, project personnel need to receive training in other areas, especially since most of the current technical and professional staff have backgrounds in the area of the agricultural sciences.
6. Training demands are dictated by a project’s features, and a feature commonly observed is the need (not always acknowledged) to have monitoring and evaluation systems in place. Indeed, training priorities should be set with a view towards achieving effective project management. Project start-up dates and extensions are also an important factor in defining training and technical-assistance needs from PROCASUR. Information on updated project plans is indispensable for PROCASUR to be able to schedule its activities and to induce demand for specific training at the right time. Unfortunately, such programming is rarely observed.
7. PROCASUR responds to project needs by using local or foreign consultants, depending on availability, it being recognized that PROCASUR’s potential lies precisely in its regional coverage. This stands in contrast with the position taken by some projects, which recommend hiring local consultants because of the lower costs involved. Lastly, PROCASUR’s relationship with other regional IFAD projects—e.g., FIDAMERICA, PRORECA, the Programme for Strengthening the Regional Capacity for Evaluation of Rural Poverty-Alleviation Projects in Latin America and the Caribbean (PREVAL), and the IFAD/Regional Unit for Technical Assistance (RUTA)—have been sporadic and limited to occasions when interests overlap. The benefits of such synergic action are evident, and a joint planning effort should therefore be launched to ensure more efficient programme support and cooperation for projects.

C. Organization and Strategy

8. PROCASUR’s training strategy displays three basic characteristics: (i) a deep social sensitivity coupled with elements of business management, strategic planning and, in general, a modern vision of agriculture; (ii) the search for efficiency with a special concern for costs, so that the service offered to projects and institutions will be attractive to them and competitive with other options; and (iii) flexibility to be able to adapt to instabilities in the countries and in the projects themselves, as described in the previous section. In general, these three features are valued by all the projects, although more is always expected of the service provider.
9. The Regional Management Unit (UDR) participates actively in the identification and design cycle of each technical-assistance and training activity, and it conducts limited monitoring of implementation. These tasks have become increasingly more difficult to perform in light of the unit’s limited resources. As for organization and methods, it is surprising how PROCASUR has been able to carry out so many activities of acknowledged high quality with such a limited staff. This has been possible thanks to the expertise, will and dedication of the staff; however, there is the risk that performance could begin to suffer as a result of excess work pressure and weak organization.
10. One feature of PROCASUR’s strategy that is worth highlighting are its efforts to systematize and capitalize on experience in two areas: (i) the variety of topics for which training and technical assistance are offered, which has made it possible to provide the same service to different projects; and (ii) the systematization of experience with training methods according to the target clients. In this regard, PROCASUR has accumulated and capitalized on various approaches that have made learning activities more meaningful from the standpoint of clients’ individual needs.
11. The recent establishment of a monitoring and evaluation system has pointed up the usefulness of systematization and organization under PROCASUR. The system is adequately designed and already includes a large part of the data on activities carried out in the past, although not to the extent required for the type of analysis that would have made it possible to enhance this evaluation. A key instrument in improving programme management has been the On-line Information System for Rural Development (SILDER). This Internet-based platform of six databases and other information serves all organizations and persons interested in rural development in Latin America and the Caribbean. It includes databases on institutions, projects, individuals, publications, training materials, events and activities.
12. D. Management Outcomes

PROCASUR’s activities in support of IFAD projects in the region as well as other projects and organizations can be grouped into four categories: (i) services offered; (ii) development of training methodologies; (iii) promotion of the market for training services; and (iv) dissemination actions. IFAD projects are the main users of PROCASUR services, although the number of other clients in the portfolio has increased. The search for new clients is part of a deliberate effort to boost revenue and capitalize on experience. At the same time, it should be acknowledged that this system was selected because of the limited supply of training activities under IFAD projects.

12. In view of the aforementioned constraints, it has not been possible to carry out all the technical-assistance and training activities scheduled. Also, in many cases a long negotiation process is involved, entailing the requesting project, PROCASUR and potential suppliers. Management of a training activity can span from fifteen days to three months, depending on the nature of the requirements, the availability of instructors, country project requirements, and approval by the project’s cooperating institution.
13. The number of participant-days dropped between 1997 and 1998, owing to a slight reduction in the number of persons trained and days per event that can be ascribed partly to changes in the methodology. Country activities continued to outnumber regional activities, as a reflection of country demands. Regional initiatives have flowed mainly from an analysis of opportunities for cooperation with other institutions and IFAD projects, such as PREVAL, FIDAMERICA and the Programme for the Reinforcement of Gender Aspects in IFAD’s Projects (PROFAGEP).
14. The suppliers of training and technical assistance are distributed evenly among the PROCASUR partners, other suppliers, and UDR staff. There has been a trend towards hiring more individual suppliers in view of the lower costs and the possibility of tapping suppliers’ specific capacities. PROCASUR’s efforts to improve the quality of training have been evident. The mean quality indicator is high and has improved over time.
15. The general conclusion drawn from the analysis of the outcomes of the training actions and technical assistance is that PROCASUR has performed a productive activity; the evaluations show that the training is of good quality and is getting better; and, in general, the projects and institutions that have received support are satisfied. The analysis also shows that many lessons have been generated that should be systematized and evaluated, as part of an effort involving organization of materials, processing of information and internal analysis, and reflection with an eye to perfecting the strategy in the future. Moreover, there is an evident need to induce more solid demand for training in areas where country project capacity needs to be strengthened.
16. Two points in particular should be noted with regard to the development of training methodologies: (i) the methodology used by PROCASUR to train project technical staff; and (ii) the methodology transferred to trainees so that they in turn may utilize it in their actions to train farmers. Since PROCASUR has repeated training activities in the same topics for various projects, it has been able to perfect both the content of the courses and the methodology over time. The staff selection methodology has been very well received by the projects in the initial phase, chiefly because it allows them to identify and select suitable personnel based on project objectives and the specific responsibilities to be performed by each person.
17. Training plans—for which a specific methodology was also devised—have been prepared by almost all the projects, although they have not always been complied with. The monitoring and evaluation (M&E) training offered through a regional event and various country events does not appear to have led to the establishment of effectively operating M&E systems in all the projects, as had originally been hoped. PROCASUR and PREVAL will need to address this matter.
18. Several projects, including the Economic Development of Poor Rural Communities Project (PRODECOP) in Chile and the National Smallholder Support Project (PRONAPPA) in Uruguay, have generated valuable experience in capacity-building with grass-roots organizations. Two areas in which progress has been made with regard to training methodology are: the preparation of business plans and capacity-building to offer private services. An aspect overlooked thus far in the methodology sphere is that of how to measure the impact of training. This is of interest not only to PROCASUR but to all IFAD projects, as well as those of other agencies.
19. PROCASUR’s work in promoting a market for training services has been fruitful. PROCASUR has graduated from being a programme in which the service-providing partners held a virtual oligopoly over training activities to the current situation in which they are given equal consideration alongside other potential candidates. The service-providing partners recognize that this change was necessary, since it allows PROCASUR to respond with complete transparency to the demands of country projects without there being a perception of favoritism towards the partners. Furthermore, the trend towards selecting individual suppliers clearly indicates an effort to value individual capacities and not just the name of a supplier institution.
20. PROCASUR ensures quality control on the basis of preselection criteria for suppliers, knowledge about the individuals involved, and evaluation of training outcomes. This gives PROCASUR a comparative advantage and allows it also to perform the function of first-level certifier (by offering requesters the chance to select from among at least three alternatives for suppliers).
21. PROCASUR established the SILDER system as part of its strategy to promote the training-services market. PROCASUR’s actions to obtain this objective have included the production of various training materials, such as books, brochures and manuals, that it provides free of charge to the persons and institutions it works most with, even though it has not developed a marketing programme for such publications. As part of its dissemination activities to promote the market for training services, PROCASUR studied the possibility of producing a newsletter to be distributed among the partners and service requesters, but decided against such an initiative. During the evaluation, the projects indicated that they would like to feel that PROCASUR identified more closely with them, the idea being to establish a more continuous relationship. In this regard, an electronic newsletter might be useful.
22. While acknowledging PROCASUR’s potential role in promoting the market for training services for rural development, it is very important that the programme study how it can generate revenue from this activity. For instance, access to the SILDER system, the promotion and sale of publications, pre-training services (e.g., locating consultants), and even training by PROCASUR staff in specific cases would all need to be part of a business strategy.

E. PROCASUR’s Impact

24. The programme has had a perceived impact in five areas, which are analysed below on the basis of available information and interviews with client projects. One area of significant impact was the definition of project strategies and training plans. As a result, projects have been able to set thematic-area and methodology priorities on the basis of the specific needs of each country project and its respective clients, organizations and individual farmers.
25. PROCASUR support has contributed towards more efficient administration of project resources, although more specific indicators are still needed. This increase in efficiency is the result of more appropriate selection of personnel, the establishment of M&E systems, and the simplification of processes for hiring private technical assistance, among others. Project efficiency has also been enhanced by building capacity among private service providers in the project areas. The issue of training as a means to increase project efficiency and effectiveness is of interest both to IFAD and to the agencies responsible for implementation of the country projects. In this connection, it is recommended that the rest of the available PROCASUR data be input onto computer-readable media and analysed, and the findings disseminated in order to show how training in certain specific aspects has improved projects’ implementation capacity.
26. One of the areas that has seen the greatest impact (through national and regional training) has been the improvement in the expertise of project staff thanks to the modern vision instilled in technical staff vis-à-vis traditional topics and this staff’s contact with new topics. These efforts have included the relaunching of government agricultural extension with technical and business assistance being provided by private-sector agents, and a new relationship with actors traditionally excluded from project agendas, e.g., municipal governments, local organizations, and communities.
27. Even though PROCASUR’s work is driven by demand from country projects and institutions, its organization and accumulation of experience have enabled it to induce demand for training in new topics and through new methods. This has been achieved through dialogue with the projects during preparation of the training plans and has had a positive impact in that training activities now include such new topics as the preparation of business plans.
28. PROCASUR has notified training suppliers of the topics that are experiencing new demand, so that the appropriate expertise can be developed. PROCASUR needs to analyse the role that it will perform in the future in terms of promoting the supply of services in new thematic areas and with the requisite quality. In this regard, it should inform all the providers of the findings of its study on the quality characteristics of training. This will enable providers to know what aspects are in need of improvement in order to offer better quality service.
29. PROCASUR has acted as a catalyst of IFAD cooperation. The programme’s impact in this area lies in its contribution to achieving greater efficiency in the transfer of knowledge for rural development. The impact has been felt in three spheres: (i) the transfer of experiences among country projects; (ii) the catalytic dissemination of the thematic contribution of regional projects; and (iii) the synergy provided by cofinancing. These have all contributed towards the effectiveness of IFAD cooperation. PROCASUR systematically promotes and capitalizes on the transfer of experiences between projects. In this way, training methods and lessons learned are incorporated cumulatively over time, as has occurred with the fine-tuning of the training plans and the selection of personnel. In the future, this could be adopted as well in the preparation of agribusiness plans and irrigation projects at the farm level.
30. PROCASUR has also had a positive impact on IFAD cooperation by building partnerships and undertaking specific activities with other programmes, such as PREVAL and the creation of the SILDER, which was developed in conjunction with FIDAMERICA. In this area, it is recommended that an effort be made to articulate all IFAD’s programmes in order to develop their agendas with a more explicit effort in terms of cooperation and more efficient use of resources. The synergy between IFAD programmes and projects, by way of cofinancing for training, yields a positive impact by motivating joint action. Moreover, it gives IFAD greater negotiating leverage with governments and local organizations for them to provide funding. In this regard, there is a need to define more precisely the rules of financial complementarity between programmes as an overall package of support from IFAD.

F. Progress towards Sustainability

31. The viability of an organization over time (i.e., its sustainability) hinges on a set of factors that can be grouped into the following areas: (i) technical capacity; (ii) strength of organization; (iii) existence of a market for the products and services it offers; (iv) financial stability; and (v) an enabling legal framework. PROCASUR has made reasonable progress towards strengthening its technical capacity as can be seen not only in its central team but also in the array of providers that are identified with PROCASUR. The programme’s technical capacity as a training service provider is recognized by IFAD’s projects.
32. Internal organization and institutional relationships. PROCASUR has posted many achievements despite its fragile internal organization. The programme’s future will depend strongly on whether that organization is able to ensure efficiency in the orientation and articulation of demand, the identification of suppliers, and negotiation with those suppliers. At the same time, the increase in its regional coverage and the number of requesters will mean greater pressure, which the current structure is not in a position to bear in the long run.
33. Growth of demand. The projects have been slow and inconsistent in articulating their demand, despite PROCASUR’s efforts and the projects’ own best intentions. In this regard, virtually all the projects admitted that they would have liked to have organized more cooperation actions with PROCASUR. At the same time, demand for PROCASUR’s services has expanded to other projects and institutions, and even to other countries, which attests to the recognition of PROCASUR’s role.
34. Financial stability. This is usually one of the most important indicators of an operation’s viability and it should be examined in light of the following four aspects, as a minimum: (i) operating costs, which PROCASUR has kept low over time thanks to low fixed costs and considerable cofinancing by requesters; (ii) the capital fund; (iii) cash flow; and (iv) the prospects for new revenue. The revenue structure does not reflect the original forecasts, since the failure to receive contributions of USD 180 000 from the European Union and USD 150 000 from the CAF necessitated a significant cash and in-kind effort by the partners in the areas of outfitting, operations and training.
35. Legal considerations. Under its mandate, the programme has carried out various activities geared towards institutional consolidation from a legal standpoint. At its June 1999 meeting, the Regional Managing Council (CDR) approved the creation of the PROCASUR Corporation and authorized the UDR to proceed with the necessary formalities without delay. In anticipation of the contribution from IFAD, a proposed business plan was drawn up for 1999-2000 (previously approved by the CDR) that considered the possibility of working with at least six country projects: the Rural Development Project for the North-Eastern Provinces (PRODERNEA) in Argentina, the Small Farmers Technical Assistance Services Project (PROSAT) in Bolivia, the Camelid Producers Development Project in the Andean High Plateau (UNEPCA), a credit project in Paraguay, Management of Natural Resources in the Southern Highlands Project (MARENASS) in Peru, and the Development of the Puno-Cusco Corridor Project (CORREDOR). The other IFAD-supported country projects did not request services from PROCASUR. In light of the findings of the present evaluation and the acknowledgement of the common constraints encountered by the projects, IFAD should ask—and encourage—the projects to reconsider their training requirements and the services of PROCASUR.
36. Outlook for the PROCASUR Corporation. The training needs of rural-development organizations and projects are destined to grow and change their content. If these needs are met, such organizations and projects will be able to perform a more effective role. These needs will grow as new paradigms are generated and as governments assume new responsibilities in addressing issues of rural poverty. Against this backdrop, training needs will become increasingly more evident. The question is whether these needs will take the form of effective demand, that is to say, the hiring of training services. This will depend, on the one hand, on an acknowledgement of projects’ capacity limitations and on government efforts to allocate the necessary resources, and, on the other, on IFAD’s effort to motivate the articulation of demand by underscoring the importance of such an investment, and to guarantee the quality of the supply of training.
37. If the demand for training services for the management of rural development increases, it is to be expected that other actors (including service companies, non-governmental organizations, universities, etc.) would be interested in becoming providers of such services. In that case, the role of an organization like PROCASUR will be very important in promoting the development of the market for training services by lowering transaction costs and making for more effective transfer of knowledge. This responsibility will be contingent upon having an organization with sufficient capacity to induce demand, to act as intermediary with responsibility for quality control, and to encourage development of the supply.

G. Conclusions and Recommendations

38. PROCASUR has fulfilled its function in terms of delivery of training services and technical assistance to IFAD’s rural development projects in the countries of the region and in others. It has accomplished the objectives identified at the beginning of the programme in August 1996. The outcomes and impact have been positive and would have been even more so had the projects received the financial backing of the countries as originally planned, in a timely fashion so as to articulate their training demands. The main outcomes were as follows:

a. The training and technical assistance offered respond to the demands of the country projects—indeed, in some cases demand was induced through training plans—to build the capacity of the projects and national and local organizations in areas originally not anticipated. The training and assistance received are of high quality, which is constantly improving and, generally speaking, is viewed positively by clients.

39. The contributions made to the development of methodologies have been significant and useful for the projects, and they attest to the soundness of the decision to include this component in the programme design. In some cases (e.g., the staff selection methodologies), the contribution was very innovative and effective in ensuring that country projects did not suffer from inadequate staffing; other times (e.g., in the case of intermittent training), improvements were posted in terms of learning levels and cost reductions.
40. In the promotion of the market for training services, significant progress was made by way of the growing link between requesters and providers of training services. Even so, PROCASUR has considerable untapped potential that could be harnessed if it fine-tunes its strategy and acts more specifically as an intermediary with responsibility for promotion and quality control, stimulating demand and the supply while seeking to keep transaction costs down in the training services market.
41. In the institutional sphere, the programme succeeded in establishing the PROCASUR Corporation, it has maintained the Regional Training Support Fund, it has created a demand that is growing, and it has its own technical capacity. An additional organizational effort is required in order to rise to the challenges of becoming an institution capable of responding to the demands of an ever more competitive services market.
42. The programme’s organization, however, continues to be weak. Its M&E system, promotion strategy, action as intermediary, and relations with the country projects need to be raised to the level of an organization that is facing growing competition and demands from its clients.
43. The following five recommendations have been formulated with an initial focus on PROCASUR and IFAD:

i. Fine-tune the strategy on the basis of an analysis of experience in order to specify more clearly the functions it is to perform, the means to be adopted, the plan for capturing and keeping clients, and financial stability. An internal seminar, properly guided, could be the means for this task.

44. Expand geographical coverage to other countries in South America, involve more service providers, present the product to more requesters, especially more IFAD projects.
45. Define itself more specifically as an organization that promotes the training services market, making intensive use of the SILDER system and other instruments currently available.
46. Adopt a business-oriented management style, maintaining interest for current immediate clients and end clients, i.e., rural development projects and small farmers.
47. Complete the work to systematize information on experiences, in particular making continuous use of the M&E system with an eye to enhancing efficiency.
48. The following additional recommendations are directed specifically at IFAD:

i. Provide financial support for the PROCASUR Corporation initiative on the basis of the acknowledged contribution already made towards improving country project capacities, and on the basis of a specific work plan to support these projects over the next three years.

2. Promote a corporative effort among IFAD programmes built around greater reciprocal knowledge of what each programme does, its work plans and the demands of the countries.
3. Promote project demand for training by contacting national authorities directly to inform them of the benefits in terms of improved project efficiency and effectiveness.
4. Stress, with the projects, the importance of enhancing their management capacity and of having sufficient high-level human resources with access to M&E systems that are in place and used on an ongoing basis.
5. Promote initiatives similar to PROCASUR in other regions, e.g., Central America, the Caribbean and Africa.

49. By following these recommendations, IFAD would use this and other regional programmes to disseminate its cross-cutting approaches, policies and strategies (e.g., gender, desertification, private technical-assistance markets, training in accounting procedures, intensive use of the logical framework, etc.), inviting projects and cooperating institutions to make intensive use of these programmes and meeting regularly with programme managers to update them on IFAD’s orientations.
50. The conclusions and recommendations made here and throughout this document are intended to provide input for high-level decision-making at the PROCASUR Corporation and IFAD as to the programme’s future. In closing, it is recommended that this evaluation be discussed by the staff of IFAD and PROCASUR in order to enrich the contributions, capitalize on the lessons learned, and formulate pertinent recommendations to IFAD’s authorities.