Interim evaluation

Introduction

This document contains an executive summary1 of the interim evaluation of the Community Development Project for the Rio Gavião Region (PROGAVIÃO). The evaluation – conducted in the state of Bahia , Brazil , in May-September 2003 – focused on analysing the sustainability of the rural development process launched by PROGAVIÃO. The possible execution of a second phase of PROGAVIÃO should be examined in light of these findings.

The PROGAVIÃO project was approved by IFAD's Executive Board in December 1995. In May 2003 the project was completing its sixth year of implementation since start-up in February 1997. The total project cost as estimated in the design was roughly USD 40.4 million, of which IFAD would finance approximately USD 20.1 million through a loan, the Government would provide USD 19.9 million and the beneficiaries would provide USD 0.4 million. The cooperating institution is the United Nations Office for Project Services (UNOPS).

The implementing agency is the Regional Action and Development Agency (CAR) of the Planning, Science and Technology Secretariat of the State of Bahia (SEPLANTEC).

Main design features and implementation outcomes 

The Rio Gavião region is situated in the southern central part of the Brazilian state of Bahia , on the edge of the so-called "Drought Polygon" of North-East Brazil . It is characterized by very harsh environmental conditions and a historical and socio-economic setting that has led to extensive rural poverty. The project area covers a surface of 11 718 km² and is spread out over 13 municipalities (municípios) . Of the total population of 40 000 families, 70% live in rural areas.

The project's formulation observed the following strategic thrusts: (a) strengthening and/or creation of rural grass-roots organizations and improvement of farmer participation – this was the overarching strategic guideline of the project and it underlay all the actions taken; (b) the importance of the relation between development and women in the rural milieu; (c) reduction of the risks associated with drought; and (d) development of agricultural production.

The general objective of the project is to increase the income and improve the living conditions of the rural poor and residents of the semi-arid region in the Rio Gavião basin within a strategy of environmentally sustainable development. The specific objectives are: (i) support for farmer participation and development of rural organizations; (ii) financing and technical support for building small-scale on-farm irrigation works; (iii) improved social conditions and local infrastructure for the education of young people by building new schools and awarding study grants, building cisterns for home use, building roads and providing rural electrification; and (iv) higher farm and non-farm income for families by means of training, extension, marketing support and credit.

The basic ecosystem of the region is the sertão2, with its characteristic vegetation the caatinga3 . The climate is essentially tropical, although it ranges from humid to semi-arid. Small holdings are the rule in the project area (85%), and tenancy is well defined. Ninety per cent of holdings are smaller than 100 ha; a survey conducted in 1998 found the average farm size to be on the order of 30 ha. There are no community-held lands.

The small production units draw on family labour, produce crops for self-consumption (maize, beans and cassava), and raise livestock on a small scale for the market. Processing of agricultural raw materials is very common in the area, e.g. cassava flour, distilled sugar cane syrup, solidified sugar cane syrup, soft and hard cheeses, and rolled tobacco leaves. Non-farm revenue – from temporary migration of males to large urban centres – coupled with other local income and social security (pension, etc.) supplement area residents' farm revenue.

Roughly 65% of the families lived below the poverty line, with an annual family income of less than USD 1 500. Women play a crucial role in this harsh environment, performing tasks such as ensuring the water supply for the household and heading up the household when the men migrate temporarily to the cities.

According to the project organization set forth in the original design, the Regional Action and Development Agency (CAR) would be the implementing agency. The project execution unit (PEU) would coordinate the field work of over 30 professionals from the expected main partners: Bahia Association of Community Schools and Rural Families (AECOFABA), National Microenterprise and Small Business Support Agency (EBRAE), State University of South-Western Bahia (UESB), and the Bahia Development Bank (DESENBANCO).

Additionally, the appraisal evaluation called for a complex management and coordination structure made up of a directing council, an interinstitutional coordination committee, municipal coordination and implementation committees, and a consultative committee.

The actual organization of the project was modified significantly vis-à-vis the design, both internally and in terms of the partner and coexecuting organizations. All the management and coordination structures described in the preceding paragraph were not implemented.

The PEU currently comprises a team of eight professionals and technicians and two secretaries (who are staff members of the CAR), plus a team of 32 professionals and technicians and seven administrative assistants, distributed among five field offices. Project execution is conducted through contracts with the following partner organizations: AECOFABA, National Agricultural Research Board EMBRAPA), SEBRAE, National Rural Training Agency (SENAR), BAHIAPESCA S/N and Bank of the Northeast of Brazil (BNB); AECOFABA, in turn, has subcontracted the Rural Cooperative of South-Western Bahia (COOPERSUBA).

Main outcomes

The project has strengthened the 80 existing community associations and established 122 new ones. These 202 associations enjoy legal status, regular attendance at meetings and monthly contributions from members. Over 190 groups have been set up within the associations to provide training in agricultural practices and group work; 153 special interest groups (both individual and group, e.g. commissions and microentrepreneurs) have been set up as a result of this support for rural organization. There are also 15 groups specifically created by – and made up of – women.

The project supported the construction of 3 680 cisterns for home use, medium-size reservoirs for each of 120 communities, seven artesian wells and 14 centralized water supply systems. It also improved the distribution of and access to electric power, with 2 492 km of lines, home connections and the distribution of solar energy kits that made it possible to provide services to 3 400 families. Other small-scale works that had considerable social impact were the construction of 36 community laundries, 67 residential sanitary services and 60 community watering points for livestock.

Livestock activities focused on the area's three main species: cattle, goats and sheep. Aside from the improved water supply mentioned above, the production of forage was the most significant outcome.

Other activities in this sector involved the use of credit to purchase animals.

In the self-supply/subsistence subsystem, implementation focused on enhancing the productivity and processing of cassava for human and animal consumption, including the post-harvest processing typical of this crop. Aquaculture – promoted under the project – is a new activity in the region and is possible thanks to the innovation of reservoirs in the arid North-East. The project is working with 144 families in 48 communities in the processing of raw materials, and with 29 collective microenterprises that involve over 855 area residents (39% women), currently at various stages of establishment.

A total of 1 739 credit operations have been carried out, principally for investments in forage, purchase of breeder stock, livestock management infrastructure and forage chopping machines. Credit has been part of the process of technical assistance and group management under very close supervision.

Some 1 400 training events have been conducted under the project, with an average of 23 participants per event. Over 7 300 participants (32% women) have attended 539 courses on rural organization and management of associations; 40% of them (2 968) also attended courses on rural administration. The project's work with the family farm schools (EFAs) has brought an additional 238 boys and girls into these schools; thanks to the study grants provided, post-primary education has now been extended to 483 young people between the ages of 13 and 17. The two schools in Licínio de Almeida and Mortugaba have been expanded, and a new school was opened in Anagé, thus creating an additional 238 school places in the area.

Project performance

The objectives as set are attainable by the project, considering that thus far nearly 9 000 families have organized in community associations or special interest groups; some 5 000 area residents have been introduced to improved farming practices, inputs and modalities; and some 3 000 area residents have received training in new areas of agricultural entrepreneurship (2 968) and/or in improved food processing techniques (3 180).

The following negative aspects have had or are having an impact on the project's efficiency, i.e. the cost-effectiveness of attaining its objectives: (a) the delay in implementation and the duration of the first phase (over three years), with weak results and constant changes in direction; (b) the location of the coordination unit very far from the project area; (c) the delay in the launching of some very important components, such as microenterprises, which is still at an early stage and has entailed considerable costs; (d) the non-implementation of other important components, such as marketing; and (e) the weak results obtained in the irrigated crops sub-component.

The strong points include: (a) the appropriate construction and intensive use of infrastructure built; (b) the results of the cassava, forage and livestock programme; and (c) the operation of the credit programme by the BNB.

The project implementation cycle is currently at a stage of maturation, consolidation and fine-tuning of the implementation actions. As a result of the delays encountered and the activities that were not implemented, and in line with good administrative practices, funds are currently available to cover a two-year extension that could include the necessary activities and thrusts of work.

Moreover, there are a number of areas for potential improvement in the project's effectiveness.

Impact on rural poverty

The reservoirs have benefited over 7 850 families; the washing points and water points for livestock – which no longer contaminate the water supply – have yielded additional benefits for families. More than 9 350 families have benefited from investments that have boosted the production and productivity of their small herds of cattle, sheep and goats.

Among the intangible individual and community assets are the new technologies applied and adopted for cassava, forage, livestock management, vegetable crops, aquaculture, water resource management, and post-harvest processing. The knowledge and skills imparted and acquired during training – which rural residents value – is another key asset generated by the project. The attendance at the courses is an indicator of its magnitude.

Family nutritional status has improved overall thanks to the greater availability and better quality of drinking water in the home. Other nutritional improvements may have been generated by the increased volume of food produced in the countryside and higher milk production as a result of better feed and drinking water for livestock; in some areas of the project, the growing of vegetables has enriched the rural diet.

The greatest impact on food security was produced by the work done in the growing and processing of cassava, a staple of the diet and economy of area families. Agricultural revenue may improve with the expansion of the livestock herd and increase in its productivity, which is occurring as a result of the adoption of a number of technological advances. An outgrowth of the concept of food security, which bears special importance in this region, is that of water security, in other words, the ability of homes and communities to ensure the supply of drinking water for basic minimum consumption for everyone at all times. In this regard, the project has achieved its greatest impact and greatest coverage.

The health conditions of the population have improved with the increased availability of safe water provided by the cisterns and the 120 reservoirs distributed across the project area. Children, in particular, have benefited from this action.

The work with the EFAs has created 238 additional spaces in these schools, providing study grants to a total of 483 boys and girls. Among the young people, girls in particular have benefited – in the best of cases, they only receive the first four years of basic education. The number of girls enrolled in the EFAs has risen from 40 to 111.

The negative environmental impact generated by the project thus far is not significant and is easily reversible through simple measures that have begun to be implemented. It will be necessary to continue observing and analysing the development of livestock herds, especially goats, and their impact on the native vegetation " caatinga" . In general, however, the project should pursue a more pro-active policy in relation to environmental conservation.

Women's presence in all areas of work and training under the project is on average 30% or more, in accordance with the projections of the appraisal report; in several cases, women have become the main players in the activities and groups promoted by the project.

The current situation of the project's sustainability does not yet provide a guarantee for its future course. Community development is at an intermediate stage of growth and must still be consolidated and allowed to mature; this will gradually reduce the paternalist vision of the project, allowing organizations to gain autonomy and build self-esteem. The project's linkages with local society, or rather its relative isolation, is the main drawback for the future sustainability of project actions.

. Viewing all these elements together, it can be said that the project has had a promising and favourable impact on reducing rural poverty in the Gavião River region.

Performance of the partners

The appraisal report did not adequately identify and assess the partners in terms of organization of project execution. The proposed PEU was too small and was expected to work with several decision-making and coordination levels that were relatively complex. The organization identified to handle all the field work and provide social and technical assistance in a decentralized fashion (AECOFABA) had no previous experience in this area. Other partner agencies identified in the design were excluded at the outset. This led to excessive complications and delays in implementation.

The cooperating institution was UNOPS, which has kept the same supervision officer since project start-up, conducting many missions and drafting the respective reports during the implementation period.

The reports show a very good knowledge of the project, the technical team, the institutional and political context, and the technical aspects involved in the productive components. The general strategy of facilitating implementation did not question, however, the adaptative changes or other, very significant changes that have been implemented.

Performance of the Government and its institutions. It was not necessary to set up any new institutional structure to implement PROGAVIÃO. The project's central coordination was placed under the CAR, a specialized agency of the State of Bahia experienced in carrying out projects with governmental or multilateral financing.

Three important government agencies took full part as coexecuting agencies since the outset of the project: EMBRAPA, SEBRAE and SENAR. In general, the contributions of the State of Bahia and other counterpart agencies (e.g. BNB) have observed the time frames and amounts required in the loan agreement. Administrative procedures have kept pace with project needs.

Performance of NGOs and grass-roots organizations. The project has wagered heavily on an NGO specialized in the development of rural alternative schools (AECOFABA), which existed before implementation and has the status of coexecuting agency for PROGAVIÃO. Its linkage with the project was focused in three areas, of which the first two had strategic value for attaining the proposed objectives: (a) the operation of three family farm schools (EFAs) in the project area, and the possibility of providing basic education to poor rural children; (b) the provision of middle-level technicians trained at its schools for the technical teams for field work; and (c) administrative facilitation for hiring all field personnel.

BNB is the coexecuting agency for the credit services component. It has developed very efficient working arrangements that are decentralized geographically and facilitate access of rural producers to credit. Efficient cooperation procedures have been put in place with the project to manage credit.

General evaluation of the project and conclusions

Design and implementation. The major thrusts of the strategy as established in the design and in the appraisal report were adequate and effective in terms of IFAD's mission and institutional vision, and ensured appropriate participation by the rural poor and other partners. However, the organizational structure established was not viable and required numerous changes in implementation, in the planned coexecuting agencies, in the management structure and in the spaces for farmer participation in the general orientation of the project.

Impact on social capital and empowerment. After an initial implementation phase of roughly three years, with a clear focus on implementation of the scheduled works in order to produce visible results, the project emphasis was shifted to community organization and strengthening of rural participation, the gender perspective, and inclusion of families in various activities.

This second stage, which has lasted a bit over three years, has shown progress in terms of better organization and participation.

The mission found that the project is now in a phase of consolidation and fine-tuning of implementation in order to attain its original objectives. Additional time is required in order to consolidate the recent, positive trends, include more beneficiaries in the project organizations and benefits, and better situate communities in the local and regional setting.

Impact on water security. The project measures to achieve water security for producers and communities have all been a genuine success. More water is now available for use in the home and for livestock, thus facilitating work in the home and alleviating the work load, especially of women and young people.

Impact on food security. The greatest impact on the system of self-consumption of the family farm unit was produced by the work done in the growing and processing of cassava, a staple of the family diet. Agricultural revenue may improve with the expansion of the livestock herd and increase in its productivity, which is occurring as a result of the adoption of a number of improved technologies.

Gender. In order to identify women's specific needs and meet their demand for goods and services, participatory planning arrangements need to be fine-tuned so as to allow for differentiated identification of the needs of each interest group in the community, and the insertion and prioritization of these needs within the work plan.

Technical staff. The devotion, expertise and enthusiasm of the field staff are undeniable. Even so, responsibilities need to be redistributed among field technicians and social assistants; technical staff should receive training in participatory planning and evaluation methodologies; and women should gradually be included more in project management.

Monitoring and evaluation. Work remains to be done in this area, especially in harmonizing criteria for system inputs; verifying, harmonizing and expanding current data; and following up on the baseline studies (profiles). The SACC system of the CAR may be a very valuable programme to facilitate this work.

Training and education . No exact information is available on the effectiveness of the training provided under the project. Under the EFA support component, it is necessary to verify if the EFAs are now better able to offer more and better services to the communities and young people they serve, and to verify the impact and changes obtained in the management of production by the 230 young people and their families who have graduated under this arrangement.

Microenterprise development. The microenterprise component and the Microenterprises and Artisans Support Fund (FAMA) are in the initial phase of implementation. A major effort – that could be characterized as preinvestment for the component – has been made in the training and organization of entrepreneurial groups in various phases of technical assistance, such as market research, definition of artisanal-industrial processes, analysis of legal provisions that are the portals of entry to the most significant markets, health regulations, etc.

The sustainability of the project achievements is not yet ensured, and an extension of the implementation period is required so that the communities can continue their processes of learning and expansion of social capital in order to strengthen their autonomy and capacity to remain active in the post-project phase.

Performance of the partners. It is necessary to conduct a joint assessment with each of the partner organizations and to adjust or modify the respective strategies in light of the results achieved and the project's objectives in order to ensure the effectiveness of the last phase and shore up or extend the achievements sought.

Insights and recommendations

It is recommended that the project be extended for an additional period of two years. Resources are available from the original agreement amount – both from the IFAD loan and from the counterpart contribution of the State of Bahia – to finance such an extension.

The country has recently witnessed a period of sweeping change at the level of the federal government, which has assigned high priority to support and integration of the poorer social sectors, family agriculture, and those regions of the country (including the North-East) with higher levels of poverty and difficulties in attaining sustainable development. Accordingly, the project needs urgently to be redefined in an effort to seek greater local, regional and social integration and to overcome the marked isolation that still characterizes it.

The development of rural community associations and effective participation by their members should be strengthened, promoting activities for: (a) territorial integration of communities – at both the local and regional level – to allow them to participate in the municipal committees and in the various projects under way in the area; (b) extension of the project actions to most of the families in the communities currently covered by the project; and (c) various actions in the areas of productive development, post-harvest processing and marketing, especially suited to collective action.

To deepen its community work, the project should revisit its structure of technical human resources and reallocate them to the new tasks while retraining staff in accordance with the new guideline that is to be prioritized.

Some specific areas of the project should be reviewed and reformulated, specifically gender, environment and M&E activities. In other activities, e.g. marketing and post-harvest processing, all the steps called for in the formulation are pending. In line with this action, specialized technical staff should be proposed for each of these areas to assume responsibility that today does not exist or is assigned to coordinators having various other responsibilities.

During this final stage, the technical staff of the project and of the coexecuting agencies should receive additional moral support and training to offset the fatigue and lag in expertise, as well as the significant deterioration of their real incomes.

Joint assessments should be conducted with the institutions associated with execution in order to adjust or formulate the respective strategies in light of the results achieved and the project's objectives.

Looking back on project implementation, IFAD did not give proper consideration to the modifications introduced in execution vis-à-vis the proposal as originally approved. Although there was close monitoring by the cooperating institution (UNOPS), the approved modifications took on a continuous and sequential character, losing sight of the deep qualitative changes resulting from their accumulation and the strategic digressions that ensued.

Future outlook. A second stage limited to the same beneficiaries would probably not yield the same levels of effectiveness in terms of marginal outcomes and, moreover, would not be equitable vis-à-vis the extensive requirements of other municípios in the region.

PROGAVIÃO has confirmed the forward-looking strategy of combining public and private efforts to address critical infrastructure issues that have a key impact on the quality of life of the rural poor.4

At the same time, it has pursued actions to empower the rural poor by organizing communities and ensuring their participation in oversight and ownership of these processes of sustainable development.

Any new project should intervene in an appropriate area of Bahia having a baseline situation very similar to the one encountered at the beginning of PROGAVIÃO. Under such conditions, it would be possible to tap the project's positive implementation experience, as well as the institutional capacity and technical human resources generated during the period.

---

1/ The interim evaluation mission was composed of: Mr Carlos Pérez Arrarte, Mission Leader, expert in production, extension and marketing; Mr Edson Teófilo, expert in policies and institutions, and community organization; Mrs Myra Speelmans, expert in microenterprises, organization and management, M&E and gender; and Mr Hernando Durana, expert in rural infrastructures and environment. The Mission remained in the State of Bahia from 11-30 May 2003. Mr.Paolo Silveri, Evaluation Officer of IFAD's Office of Evaluation joined the mission at the beginning and at the end of the field work, and the Supervisor of UNOPS for PROGAVIÃO, Mr Rodolfo Lauritto, joined the Mission in its first week of work.

2/Dry hinterland. Typical of the Northeastern region of Brazil .

3/ Indigenous word which means "white bush" - a natural vegetation of the semi-arid region of Brazil .

4/For instance, those that establish water security for residents and their livestock.