Income - Generating Activities Project (2005)
The Benin Income-Generating Activities Project (IGAP), approved by IFAD's Executive Board in December 1995, formally started on 13 September 1996 with the signing of the Government of Benin and IFAD of a loan agreement in the amount of SDR 8.05 million (equivalent to USD 12 million); the project staff was recruited in December 1996. The planned duration was seven years, with a completion date of 31 December 2003.
With a view to the coming end date of the project, an IFAD interim evaluation was carried out in May-June 2003. According to the Approach Paper for the evaluation, its ultimate objective was to assess whether the results, impact and efficiency of the project justified a continuation or whether a different approach should be pursued, if the decision-makers should agree to continue it.
Given the constraints and weaknesses facing the rural poor in Benin (identified during missions and studies conducted 1993 to 1995), the IGAP pursued a multi-sector strategy for working with the rural poor in the four départements in southern Benin 1. The total project area is 24 000 km2, and the total population in the project area is 2.4 million, excluding Cotonou and Porto Novo. This substantial demographic pressure, coupled with the small plot sizes and permanent land use, are the main causes of land degradation, erosion, loss of fertility and low yield levels.
The development objective of the IGAP is to increase income and improve food security for the target group. The immediate objectives were to develop income-generating activities, protect the environment and promote grass-roots institutions.2
The project had originally been designed with two components: income-generating activities and support for grass-roots institutions. However, the objective of the second component was multi-faceted and very broad in scope, ranging from "ensuring the sustainability of activities under the income-generating activities component" 3 to "promoting grass-roots participation, at all levels, so the rural community would eventually be able to take ownership and assume responsibility in the areas that are instrumental to sustainable development." 4 Therefore, the two original components were divided into three when the project was launched:
i) an IGA-component seeking to "create a network of economic (micro-)enterprises formed either by groups or by individuals (micro-entrepreneurs) who belong to the target group".5 As per the logical framework, however, the micro-enterprises targeted were basically those made up of groups. The IGAs were to be supported by community discussions and coaching of the groups, technical training, the development and re-transfer of 150 ha of valley bottoms as rice paddies along with support for the groups to purchase equipment and 500 ha of land;
ii) a rural finance (RF) component, with a sub-component to provide direct support for establishing FSAs as well as one for a credit line and risk fund at the Fédération des Caisses d'Épargne et Crédit Agricole Mutuel (FECECAM), to grant loans for the IGAs (particularly medium- and long-term loans to buy equipment and land); and
iii) a grass-roots institution support component to foster community development by forming and strengthening village development committees (VDCs) and groups, establishing socio-economic infrastructure, providing literacy training and reinforcing national NGOs.
A final component was the establishment of a project management unit (PMU) with "full administrative and financial autonomy" 6 – recruited against precise terms of reference and documented competencies – to manage the project using the "faire-faire approach".
The "faire-faire approach" is intended to increase the effectiveness of implementation and the participation by beneficiaries (small rural producers) in defining their problems. It seeks appropriate solutions for strengthening the means available to the beneficiaries and increasing their responsibility in implementation and management of the defined development activities. Unlike the traditional "faire approach" (where the management both implements and supervises the project), the "faire-faire approach" is characterized by having the PMU give guidance and supervise only, while responsibility for implementation lies with the service providers carrying out the field activities. These service providers comprise NGOs (pre-selected to provide on-going training/coaching to the groups), micro entrepreneurs, VDCs and FSAs, and on the other side, consultants, entrepreneurs, the Regional Rural Development Centres (CARDER) to provide specialized services such as the dissemination of training, building of infrastructure and site supervision. Under this approach, the NGOs are hired on the basis of a rigorous selection process, with annual fixed-price contracts and a semi-annual performance review.
Depending on the type of services involved, the other service providers are hired on annual contracts or by the job.
After over six years of activities, the IGAP has begun to have significant poverty-reduction impacts on the target groups according to the indicators monitored, such as increased income, access to potable water, food security, acquisition of consumer goods, etc. The project also had a significant impact on the socio-economic status of women.
Among the impacts identified, some are easily attributable to the IGAP because of the exclusive nature of its work in a specific sphere or area (e.g. literacy progress in the area where the IGAP is the only project). Further, in September 2001 an assessment was made with the NGOs working in the project area of the changes attributable to the project, and a number of areas were identified in which the project had had a clear influence on the impacts observed. There also seems to be a cumulative effect of the IGAP's interventions on the impacts produced, as shown by the impact of producer training.
Within the IGAs component, the project supported the establishment of several hundreds of micro-enterprises covering over 50 types of economic activities (45 traditional and six innovative), distributed over five sectors. Paradoxically, these micro-enterprises are mainly single-proprietor enterprises, even though most of the strengthening and follow-up efforts were deployed for production groups; as a result, it is not possible to know the exact total number of micro-enterprises formed. As of 21 June 2003, a total of 386 production groups were receiving support (out of 450 projected): 62 were men's groups (16%), 200 were women's groups (52%), and 124 were mixed (32%). Women represent approximately 74% of the group members supported. Most of the groups have more than two activities and operate in more than one sector. While the potential sustainability of sole-proprietor micro-enterprises is strong, very few production groups will be able to survive, as their usefulness for most of their members is much more of a social nature than as a viable economic unit. In addition to the planned project activities to support the establishment of these micro-enterprises, the project will have created 195 food processing workshops (e.g., converting cassava into gari, pineapple into juice, etc.) in as many groups. However, the creation of such centres, on a grant basis, runs counter to the project's overall strategy for this component, which was that any action (i.e. with groups) would be undertaken on the basis of credit and not subsidies. 7 Only 93 ha of valley bottoms will have been developed by December 2003, and the acquisition of equipment and land has been quite low.
As for the rural finance component, 44 FSAs have been set up with buildings and equipment, covering roughly 320 villages, i.e. many more than the number of villages originally planned. These FSAs are held by 15 479 shareholders, 6 002 of whom are women (49% men, 39% women, 12% legal entities). However, it is estimated that the actual number of members is some 25 000, more than 50% of whom are women (bearing in mind that 75% of the members of groups and cooperatives are women, who are often not individual shareholders).
As at 31 March 2003 the capital mobilized by these 44 FSAs was CFAF 170.5 million. Although women represent 39% of the shareholders and 50% of the members, they hold just 22.2% of the capital. The FSAs have credits outstanding of CFAF 302.2 million, mainly consisting of small loans for three to six months, in other words the kind of credit that meets the needs of the poor. Since the outset, 69.5% of shareholders have received a credit. Out of the 44 FSAs, ten already have the capacity to cover their expenses completely and seven others can cover 90% of their expenses. It should be noted that the FSAs experienced a major crisis in 1999-2000, but, remarkably, were able to weather that crisis. On the other hand, to date only CFAF 65 million, or 7% of the allocated funds, were disbursed to the FECECAM credit line, which means that very little medium- and long-term credit has been granted. Use of this credit line is thus abnormally slow, and it is unlikely that the line will be fully used in the coming months.
Lastly, 72 VDCs have been formed to date, or approximately 87% of the 83 targeted villages, and the PMU planned to work on the remaining 11 in 2003. VDCs generally enjoyed a high level of acceptance by the authorities and the people in the villages as a development tool. Nonetheless, the VDCs were still quite far from being the tool that "the rural world can use to take ownership and responsibility of the areas needed for its sustainable development".
The fact is that, behind the concept of the VDC, there is a multitude of institutional realities, but their existence and function are basically determined by the existence of the IGAP. The functionality of the vast majority is rather weak, and sustainability is very uncertain, particularly in the current context of the decentralization of institutions. With regard to socio-economic infrastructure, the completion rate was generally low (16% to 42%), except for the classroom modules (which will be 90% complete). Literacy training has also produced disappointing results, with just 3 166 having received such basic training, out of a projected 11 250. The strengthening of national NGOs has yielded good results in terms of the quality of the services they provide to the IGAP, but also at the price of a number of perverse side effects on beneficiaries, the NGOs themselves and the relationship of the project with the NGOs.
As for the PMU, it was set up as planned. It is dynamic, competent, effective and thorough, while at the same time flexible, transparent and results-focused and it learns from its mistakes. Except for gender issues, the management systems in place are well tested, effective, reliable and suited for their mission. The PMU handles the faire-faire approach well, though it is quite costly and has not yielded the expected results in terms of participation and ownership by beneficiaries.
The IGAP offers good potential for the sustainability of all its impacts and outcomes. However, sustainability of these impacts could be threatened by the project's very success if steps are not taken (assuming the project is continued) to address other poverty reduction needs. The project was also able to create innovation in two of its components, with strong potential for replicability, and these innovations have already been tapped by the PROMIC (Microfinance and Marketing Project) 8.
When reviewing these outcomes, it seems that the IGA component has been both relevant and effective, though in a manner not planned by the project (i.e. for single-proprietor micro-entrepreneurs instead of production groups), while its efficiency varies depending on the type of intervention. The foreseeable sustainability of IGAs for individuals is generally good (except for enterprises requiring extended surface areas, owing to low land acquisition rates) while that for groups is almost non-existent.
Even though the FSA sub-component of the RF component got off to a very rocky start, its spectacular comeback since 2001 and the very significant outcomes to date make this sub-component very relevant, very effective and reasonably efficient. The outlook for sustainability of the FSAs is promising, but could take time and require sustained and rather long-term support. On the other hand, the FECECAM credit line sub-component has proven to be irrelevant, ineffective and consequently inefficient.
Under the grass-roots institution support component, the sub-component for strengthening NGOs lacks relevance. The relevance of the other sub-components should be re-examined, as their effectiveness and efficiency are average and their sustainability potential quite variable.
The relevance of the PMU is meaningful only if the project is continued. It is effective, but could be even more so on the gender issue by taking on a strategy and specific actions. It could also be more effective by adopting a different implementation approach (such as the faire avec approach), externalizing certain costs and reducing the number of sub-components in the project.
In light of the foregoing conclusions, there is justification for pursuing some of the components and sub-components, if at rather different levels and duration, whereas the possible continuation of the other sub-components will presume more in-depth analysis of their added value, efficiency and sustainability. If the project is to be continued, these analyses should be conducted before any extension is implemented; the "faire-faire", group and gender approaches should also be reviewed; and the formulation of any continued work should take special account of other initiatives and projects in the same or in complimentary sectors (e.g. PADSA II, the Social Change Observatory, etc.).
1/ After the project start, a new administrative division was introduced, changing the four initial départements – i.e. Mono, Atlantique, Ouémé et Zou – into seven departments, by adding Couffo, Plateau et Central.
4/ Idem, pp. 19 and 21.
7/ Appraisal Report, September 1995, Volume I, p. 18.
8/An IFAD project in the northern half of Benin that is almost identical to the Income-Generating Activities Project in terms of objectives and activities.
Peasant Development Fund Credit Project (2005)
Background. The loan for the Peasant Development Fund Credit Project – Eastern Region of Paraguay was approved by the Executive Board of the International Fund for Agricultural Development (IFAD) in December 1995. Project activities commenced in January 1996 with an implementation period of six years, which was extended twice prior to the current completion date of 31 December 2004.
Approach and methodology. The IFAD Office of Evaluation sent a mission to conduct a completion evaluation1, which visited the country from 9 to 31 March 2004. The methodology for evaluating IFAD projects, calls for close cooperation among the borrower, project management and beneficiaries to review project progress and impact together. During the evaluation process, the mission applied the methodological framework adopted by the Office of Evaluation.
In Paraguay, the mission met with authorities from the Ministry of Agriculture and from public and non-governmental implementing institutions, cooperating agencies and other key institutions, and with the IFAD Country Programme Manager. Visits were paid to the areas where the project is active, and contacts made with cooperatives and non governmental organizations (NGOs), grass-roots organizations, beneficiary groups and related parties. On 30 March 2004, the mission held a workshop to present preliminary conclusions to all stakeholders, and prepared an aide-mémoire setting forth preliminary findings and a record of proceedings.
Major design features
Project rationale and strategy. The project at conception was framed within the Government of Paraguay's rural poverty reduction and small-scale agricultural development strategies, as well as the IFAD sustainable agricultural and rural development strategies and country-specific circumstances in addressing the rural poor. In view of prior experiences, this project elected to focus on just three components: financial services, technical assistance and evaluation.
To facilitate and expand access to formal credit by small-scale producers and micro-entrepreneurs, project design called for strengthening the Peasant Development Fund (PDF) 2 and expanding and strengthening intermediary financial institutions (IFIs)3 .
Project area and target group. In establishing the project implementation area, consideration was first given to the eleven departments located in the eastern region that had not been included in the credit project for the northeastern region 4. Subsequently, the three departments covered by that project were included as well. The target group comprised 65 000 farm families with holdings of less than 20 ha and annual income below the poverty line, set at USD 3 500 per annum per family. Half of these families did not hold title to their land.
Goals, objectives and components. The project's main objectives were to improve the credit system serving farmers in the target group, and to develop and diversify the supply of financial services in rural areas. Anticipated final results were to be an increase in small-scale producers' incomes and in living standards for campesino families. The project was to benefit directly some 12 550 families (20% of the target group) and 300 small rural enterprises, which in turn would benefit the target population by purchasing agricultural products and generating rural employment. Particular attention would be given to participation by rural women. The project was to incorporate 58 IFIs, including 22 consolidated, 21 reorganizing and 11 emerging cooperatives, and four first-tier banks. Technical assistance for production was to be provided for some 4 300 producers, and assistance in obtaining property title to 1 300 campesinos.
To achieve these objectives, the project called for implementation of the following components: (a) rural financial services, to be carried out by PDF by creating a credit fund and providing institutional and operational strengthening for it; (b) technical assistance services, to be carried out through an ad hoc agency and coordinated by the Technical Assistance Coordinating Unit (TACU) to assist IFIs in managing their financial services and to assist their members in production, marketing and land titling; and (c) evaluation, including contracting for a survey and baseline study, two evaluations of implementation and specialized consultancies, and case studies on IFIs and producers' groups. Gender was not covered in a separate project component since it was considered a fundamental cross-cutting element in implementing the various components.
The total project cost was USD 18 million,5 of which IFAD was to contribute USD 10 million, the borrower USD 7.2 million and beneficiaries USD 0.8 million. The base cost for the financial services component was set at USD 10.29 million, with USD 6.27 million 6 for the technical assistance services component and USD 544 000 for the evaluation component.
Implementation arrangements and partners. PDF is responsible for overall project management and coordination and for direct implementation of the rural financial services component. Campesino organizations and cooperatives and various financial and commercial institutions could be recognized as IFIs in accordance with rules, conditions and criteria set by PDF. TACU was conceived at a higher level to coordinate technical assistance with credit. The United Nations Development Programme (UNDP), an international organization specializing in administering project implementation, was to perform a technical and administrative role in implementation of the technical assistance component. A technical and administrative team, Technical Assistance Services Unit (TASU), was set up for this purpose. Project evaluation was to be performed by an independent local institution selected under PDF criteria. The Andean Development Corporation (CAF) was responsible for supervising implementation for the first two years of the project, but was replaced by the United Nations Office for Project Services (UNOPS) in 2000.
Major political and institutional changes during implementation. The political and institutional context was marked by instability during the project implementation period. In 1998, the Government of Paraguay ceased to provide local counterpart funding committed. In 1999, unpaid loans incurred by farmers during the previous agricultural season were forgiven by law. Between 2003 and early 2004, the Government's failure to pay its debt to IFAD led to a suspension of disbursements under the loan.
In practice, project implementation presented a change in institutional arrangements, since the mechanism planned for coordinating the project components, TACU, was never fully implemented.
With respect to external institutions, the most significant change was the replacement of CAF, the initial cooperating institution, which operated from 1997 to 1999, by UNOPS, which commenced operations in 2000.
Design changes during implementation. Two changes occurred: a USD 4 million reduction in the amount of financing provided, and the incorporation of three additional departments from the previous project for the northeastern region into the project area.
Summary of implementation results
Target group income and savings. The project's impact on the incomes of its beneficiaries is negligible, since exogenous factors such as the market and policy decisions were more important. Generally speaking, no mechanisms or systems were set up to promote savings, nor were the producers interviewed predisposed in this regard 7.
Improving access to credit for the target population. Of the 12 500 targeted beneficiaries of credit and other financial services, seven years into the project only 2 271 were being served, with the highest number (3 470) having been achieved in 2002. Not all borrowers were able to adjust to the conditions imposed on project beneficiaries, and practically no women were included. In addition, the access achieved in large part assured no continuity, since most of the IFIs were considered unsustainable. All credits were agricultural, with no other financial services having been developed 8
Strengthening of the PDF-IFIs financial system. Most of the measures required to strengthen PDF such as increasing the number of people served within the target group and lowering the cost of credit-related operations were not carried out. 9
The administrative staff grew significantly while portfolio management staff remained small. The project managed to restructure some IFIs that were facing financial crises and administrative problems, and provided administrative and financial strengthening for other, either incipient or structurally weak IFIs. Nevertheless, a high number of cooperatives and associations supported and incipient IFIs do not appear sustainable.
Technical assistance for IFIs and their members. The greatest number of IFIs were assisted in 2002, when 39 IFIs participated and close to 3 500 beneficiaries received production assistance. Financial services were provided by the project in 2003 for 23 IFIs of the 58 planned for consolidation years. The number of IFIs incorporated during the project's life up to 2003 was 42, but assistance was either suspended or terminated for 19 of them. The evaluation showed little transfer of skills to IFI management and membership through technical assistance provided under the project. Plans called for providing technical assistance to 4 394 small farmers for production. By the end of 2003 about 3 000 producers were being supported, some without credit. Some IFIs took initiatives or expanded their services by channelling the supply of inputs and/or marketing of some agricultural products to beneficiaries.
Assistance to land titling services. Aside from one attempt at the outset of implementation, no results were reported with respect to the 1 300 beneficiaries planned.
Gender focus. None was implemented at the outset (not until 2001). The most obvious adverse result was the exclusion of women as a priority target group, and their minimal or non-existent participation as beneficiaries of credit and technical assistance.
Establishment of the Technical Assistance Coordinating Unit. Despite some initial attempts and fleeting periods of operation, the TACU was never set up or operated as contemplated in the project. This affected coordination of technical assistance and financial services.
Relevance of objectives. The poverty prevailing among Paraguay's rural population fully justified the project's overall objective. Moreover, the weaknesses of the rural financial system, together with a lack of knowledge of appropriate techniques and practices for diversification, crop improvement and marketing among small farmers, placed severe constraints on the development of productive activity, lending priority to the objective of improving access to formal credit and other financial services by the rural poor. In addition, instrumental objectives such as strengthening of PDF and IFIs,10 improving productive capacity, preserving natural resources and regularizing land tenure, favour access to and better use of credit resources.
Effectiveness. An analysis of the actions and results of project implementation shows that the project was not very effective in achieving either the ultimate objective and strategic objectives, or specific goals for each component. Physical and financial targets in terms of the number of beneficiaries are far from having been reached.11 Moreover, a bias in credit allocation was apparent towards neighbouring areas with the highest number of target population.12 Although a substantial number of IFIs were covered, some for a short period of time, barely 20% of targeted borrowers were reached. No credit was provided for micro-enterprises, nor for women and young people, nor for solidary groups, all of which were called for in the project design; 13 while the savings mechanisms and other financial services provided for exist only by exception. PDF did not implement the risk coverage plan to enable poor rural settlers with little or no collateral security to join IFIs.
14 The IFIs in turn did not implement the contemplated system of specialized credit assistants providing for outreach by IFIs to the poorest. The project performance does show improvements in credit administration by IFIs. Similarly, technical assistance for production offers some notable results.
One factor that may have had an adverse effect on the project's effectiveness is the fact that credit was almost always provided before technical assistance services, even for incipient IFIs.15
The failure to consolidate the TACU as a coordinating unit gave rise to separate operations by the two components, jeopardizing the effectiveness of their results.
Efficiency. PDF did not only lower the administrative cost of credit but increased it 16,and presents excessive growth in arrears. There are signs that funds were diverted and misused in managing PDF; some of these were noted in supervision reports.
One factor that had an adverse effect on TASU operations had to do with the instability generated by uncertainty and delays in the flow of budgeted resources.
The lack of a baseline study and an adequate monitoring system, in addition to the failure to comply with the evaluation and case studies, makes it difficult to conduct a review with broad coverage and accuracy in measuring efficiency. Nevertheless, the high unit cost of operations increases in arrears and poor performance in achieving targets call into question the overall efficiency of this project.
Impact on rural poverty
Impact on physical and financial assets (moderate, rating 2). Improvement in the physical capital of project families as a result of implementation is minimal. The project showed a propensity to act and intervene in areas and IFIs that were relatively well off compared to most of the target group. In these areas, some changes can be observed in physical assets in terms of improvements and/or additions to housing and improvements in land use potential. Access to financial assets improved moderately with the project, but savings and capitalization are negligible. Some project beneficiaries purchased livestock, particularly animals for fattening and small livestock, which serve as a reserve or kind of savings for farmer populations. Less frequently, additional plots of land were purchased.
Impact on human assets (from negligible to moderate, rating 1-2). Training for staff in project implementation units was generally weak and unsystematic. The programme to train management and administrative staff in IFIs, and technical assistance provided to producers who are members of IFIs, have contributed to human capital formation in some groups of the target population, but its impact has been limited by a lack of continuity and overall vision of events and assistance, as well as an almost total exclusion of women and young people.
Impact on social capital and empowerment of the population's capacity for action (moderate, rating 2). Despite the project's significant contribution of promotion, training and assistance for IFIs, a lack of attention to effective participation by members and effective social control of activities have in many cases undermined members' trust and expectations.
Impact on food security (moderate to negligible, rating 2-1). No severe deficiencies were observed in the nutritional status of children in the area 17. The project may be making a modest contribution through income, diversification of production and assistance.
Impact on institutions, policies and regulatory framework (from negligible to moderate, rating 1-2). The project has had little institutional impact. Even the expected impact on strengthening PDF did not materialize, in regulations, procedures, efficiency or transparency of operations. Nor did TACU manage to operate as a coordinating unit, involving the Ministry of Agriculture albeit indirectly.
Impact on women (negligible, rating 1). The project's results in terms of including women in the target population and in actions and decisions relating to implementation have been very limited19, and in some cases generated conditions that increased the workload of women and children.
Sustainability (very low to low potential, rating 1 to 2). The benefits generated by the project have to do with the workings of public and private institutions created or strengthened by it. Sustainable institutions are important to ensure the continuity of such benefits. PDF's financial statements reflect considerable equity capital and an increase in profitability 20, but do not take into account the substantial contributions received from the Government, including from Inter-American Development Bank (IDB) and IFAD loans, nor that a high percentage of income (averaging 32% since 1996) come from investing part of those funds separately from its loan portfolio in time deposits or savings, so that they are not available to serve demand for financing from farmers. 21. In practice, as indicated throughout the report, PDF presents severe functional weaknesses, high costs per unit of credit and an increase in arrears that gives cause for concern.
Most of the IFIs supported by the project are not sustainable over the long term. Many of them have been structured to provide credit to their members, some specifically with PDF resources. They depend on external funding sources (governmental or other donors) because their members do not make savings deposits; their managers generally possess limited management skills; member participation is low and member knowledge of IFI operations generally inadequate. Furthermore, IFIs in general lack appropriate procedures to mitigate credit risks, as well as collection and recovery policies and procedures.
TASU, which provided technical assistance services, was never contemplated as a permanent institution, although beneficiaries were to contribute to payment for such services. The IFIs and beneficiaries interviewed by the mission generally value such services, but it is highly unlikely that they or small-scale producers are prepared or able to take over financing of their own operations.22
Innovation and scalability (moderate, rating 2). The project called for the creation of a financial system oriented towards small and very small-scale producers, with a second-tier institution and a number of associations and cooperatives consisting wholly or in part of rural population groups acting as financial intermediaries. At the time this was a promising innovation for the country, and it replicated and expanded upon the project implemented in the north eastern region (three to 14 departments). The fact that implementation has not been successful does not invalidate its potential for replicability given the necessary organizational, functional and operational adjustments and more stable conditions.
Other kinds of impact on poverty: Impact on agricultural production and productivity (moderate, rating 2). Some improvements in production and productivity can be observed. Technical assistance for production provided by TASU is recent and not always tied to credit, but the mission observed some progress. Although technical assistance under the project promotes crop diversification, there is an observable trend towards intensification of cash crops. Irrigation is not widely used. Marketing assistance (including exports, in some cases) has had a major impact on the incomes of producers assisted.
Overall evaluation of impact (moderate, rating 2, with cases of negligible impact, rating 1). The project has had little impact on the agricultural context in rural Paraguay. One of its main objectives, to strengthen PDF, not only was not achieved but, in part for reasons beyond the project's control, backsliding and distortions occurred, with an illicit misappropriation of resources and increased bureaucracy, operating costs and arrears. An insufficient number of IFIs were incorporated, and many of these show little potential for continuity over the medium term.
Technical assistance was added late and actions at the IFI level were directed more towards helping them function or recover than towards training their management and membership. As for producers benefiting from assistance, there are fewer than planned and the assistance period has been short, so that the effects remain to be seen
Performance by partners
Performance by IFAD. The IFAD Operations Manager maintained a close working relationship with the project, supporting actions by implementing staff and helping to promote support and coordination among national and international institutions. This relationship was affected by political turbulence. Supervision was stepped up in 2000 and several support missions were organized; however, the lack of attention to major observations and recommendations could have benefited from clearer decision-making.
It is important to recognize that both IFAD and UNOPS had a positive influence on maintaining a solvent portfolio, even in the face of widespread payment default following the debt cancellation initiative. Nevertheless, insufficient attention paid by the national institutions involved to repeated observations and recommendations by project supervision missions affected performance and should be given special consideration in the future.
IFAD met its financial commitments towards the project satisfactorily, including its contribution to resolving the problem caused by cessation of counterpart contributions by the Government of Paraguay. Also, IFAD granted two extensions in the project implementation period and loan closing date.
Performance by the cooperating institution. Initially, the cooperating institution was CAF. In 2000, IFAD decided to replace CAF with the United Nations office responsible for supervising projects in the region, UNOPS. The experience with UNOPS has been satisfactory and supervision of project implementation has taken place systematically and frequently 23.
Although the supervision reports have been made available with some delays,24 timely communications to the Ministry of Agriculture and the leadership of directly concerned institutions have diligently reported on major problems identified by each mission and formulated relevant recommendations.
Performance by the Government and its agencies (including project management). Political instability affected performance by the institutions responsible for implementation. Generally speaking, the project has not had the required political, programming and budgetary backing. For instance, despite the loan agreement, the Government ceased to comply with its counterpart obligations.
Although the plan was to make PDF politically independent, governed by an executive board acting on behalf of the institution and its beneficiaries rather than to the benefit of political parties and appointees, that was not the case for most of the project implementation period.
The institutional and administrative arrangements contemplated for project implementation did not materialize, and it was not possible to provide for regular TACU operations. The lack of coordination among implementing agencies, even though just two operating components were involved, led to an internal division that diminished complementarity, eliminated synergies and had a definite impact on potential for success. The unit responsible for technical assistance, TASU, made an effort to perform its functions as written, to carry out administrative actions with transparency and to provide instruments to empower beneficiaries. However, a combination of unfavourable circumstances such as the commencement of operations two years late, problems with the stability and continuity of technical assistance, complex or slow administrative processes in making loan proceeds available and delays and cessation of local counterpart contributions 25 prevented the services from attaining the expected level of effectiveness 26 and sustainability.
Performance by non-governmental and grass-roots organizations. Many of the IFIs that serve IFAD's target population exist mainly to provide credit to their members. Once the members receive the credit they have little interest in the IFI's survival, and arrears are common. As a result high arrears rates, both internally and externally, are typical of IFIs. Nor is there much interest in attracting new members or bringing in women.
Other NGOs have participated in the project. Among them are the second- and third-tier cooperatives that have provided training and support. Mention was also made of CEPACOOP (Central Paraguaya de Cooperativa), the federation of cooperatives created with support from the project that operates in agricultural product marketing and exports.27
The Paraguayan Centre for Social Studies (CPES) satisfactorily performed the first and only external evaluation of the project. Successive evaluations and case studies contemplated by the component were delayed and finally not carried out.
Overall project evaluation and conclusions
The lack of continuity in project management and executive authorities, unstable agricultural prices and policies, financial vulnerability to political and personal pressures and interests, the tradition of clientelism and paternalism within institutions, weaknesses in monitoring, supervision and administrative control over resources, operating and regulatory deficiencies in official and private organizations concerned: all these factors were present during the project implementation period and had a decisive impact on its operations and results.
The mechanism included in project design to coordinate the components, TACU, with participation by the Ministry of Agriculture and PDF, did not achieve stable operations. Coordination was weakened and the mechanism became diluted.
The fact that a high percentage of the beneficiaries served by the project did not belong to the lower strata of IFAD's target population led to a less perceptible improvement in incomes, physical and financial capital and general living conditions for that population as a whole.
In view of the foregoing considerations, the project did not reach its goals or achieve the expected results, although it made important contributions that warrant recognition.
Findings and recommendations
A minimum of continuity in institutional and sectoral policies, and in the tenure of project technicians and authorities, is a prerequisite for the success of rural development and poverty reduction projects.
To attempt to separate the administration of a public project from direction and oversight by the competent authorities does not protect the project management from the influence of politics and change. In fact, it weakens the financial and political support needed by implementing agencies and can thus impede implementation.
The project components should not be allowed to evolve independently, without practical executive direction or at least effective coordination during implementation.
Organizing associations and cooperatives motivated exclusively by the expectation of obtaining financing, without proper training in cooperation and principles of solidarity, will backfire and lead to serious losses and disturbances.
A project that includes financial services to be provided by a second-tier institution requires sound intermediaries that are sustainable over time. If solvent financial institutions such as commercial banks and consolidated cooperatives are to join the project, it is crucial to establish adequate incentives along with clear, well established operating rules.
Training within a cooperative or other grass-roots association to improve its management and member participation must be addressed to administrative staff and all members in addition to executives.
There appears to be resistance among financial institutions to contribute to developing a savings mentality among their members and provide related financial services. Such services should be emphasized at the project outset, since the evaluation shows few initiatives and accomplishments in this regard.
Supervision missions by IFAD or the cooperating institution, as well as evaluation and monitoring studies, should be systematic and timely and formulate viable recommendations. It is also important for cooperating institutions to require those implementing the project to fulfil the commitments agreed upon in documentation signed by the parties concerned.
Review of PDF structure and organization. A full-scale review of PDF structure and organization is needed, 28 taking into account scarce resources and the objective of serving a disperse population. The reengineering of PDF should focus on a higher degree of solvent and consolidated intermediary institutions; identification and promotion of innovative solutions to channel and manage credit and other financial services for small-scale producers and the rural poor in general; design and implementation of an efficient mechanism to support high-risk credit; ongoing supervision and support of weaker or incipient IFIs; a search for mechanisms and instruments to enable IFI management and members to receive organizational, cooperative, administrative and accounting training, as well as guidance in production, marketing and project development; mainstreaming women into credit; and promotion and implementation of credit for microenterprises and non-agricultural production in rural areas. At the same time, PDF should reinforce normative elements and discipline in its operations, as well as internal audit and control.
If PDF is to achieve a reasonable degree of political independence, a change in executive board membership should be considered, to include high-profile individuals with a reputation for integrity and independence, who are familiar with the financial system and microfinance. Procedures should be established to ensure that board members cannot be removed for political reasons, but they should rotate periodically.
Adopt corrective measures to secure PDF lending portfolio. (i) invest in training IFIs in group and credit management before disbursing loans; (ii) review loan approval process to ensure credit timeliness; (iii) enhance post-disbursement supervision; (iv) conduct field audits to verify methodology, policies and procedures applied by PDF and IFI staff; and (v) implement active procedures to recover arrears.
Disseminate PDF financial services. Once its lending products have been restructured, PDF should launch a large-scale information campaign to publicize its services and inform its target market of its existence and its methodology and requirements.
Incorporate consolidated IFIs. The consolidated IFIs, including both the private and official banking sector, must be motivated and have incentives to participate in projects with PDF. Also, small and incipient IFIs should be encouraged to join other, more developed ones, or to join second-tier organizations.
Provide pre-training for incipient and recovering IFIs. These IFIs require significant training in financial and credit management before receiving loans.
Focus on beneficiaries. Appropriate steps should be taken so that beneficiaries are incorporated in IFAD-supported projects in ways that are more consistent with their objectives.
Train target population. The members and management of IFIs need training in the principles of cooperation and organization, as well as intensive learning and training in credit management and administrative and financial issues, before receiving credit. Priority should be given to providing training and assistance to groups of women, young people and members generally in setting up and operating micro-enterprises and non-agricultural productive activities.
Ensure integration and complementarity between technical assistance and credit. Cropping plans and investment projects by borrowers should be well supported, based on technical and economic criteria, anticipating the marketing of production and considering flows of funds for establishing recovery periods.
Provide for continuity of technical assistance for beneficiaries. It is important for the Ministry of Agriculture to examine the need to ensure a certain continuity in technical assistance services for IFIs and borrowers under the project, considering the imminent closing date.
Designate a higher executive authority. In future projects, the competent governmental agency should designate a unit responsible for directing and coordinating implementation of the project and all its components, with sufficient backing and authority.
Require compliance with programming of evaluation and monitoring activities. Rigorous evaluation and continuous monitoring of project implementation should be assured. The evaluating institution should be appointed following a strict selection process.
IFAD participation during implementation. In countries with an unstable political situation and weak or vulnerable institutions, IFAD can play an important role in institution-building, continuity of programmes and technical staff and fulfilment of commitments set forth in loan agreements.
Strengthen participation by regional or international institutions in supervising and monitoring actions by bodies responsible for project management and implementation. The support they provide for implementation, and the support they receive from IFAD, should be reviewed in depth in order to realize the full potential of the intervention.
1/ The IFAD evaluation mission comprised: Mr. José Raúl Alegrett Ruiz, Mission Leader; Mr. Reuben Summerlin, Financial Expert; Ms. Patricia Reynoso, Sociologist; and Ms. Maria Soledad Marco, Evaluation Consultant, assisted in the implementation of the new evaluation approach and methodology. The evaluation was led by Katharina Kayser, Evaluation Officer of IFAD's Office of Evaluation.
7/ With the exception of investments in animals, housing improvements or land acquisitions. With respect to savings, there is a tendency to increase personal contributions to capital to improve the contribution-to-credit ratio, but generally savings capacity is virtually non-existent.
8/ PDF promoted production credits for rural enterprises and pre-investment services, both short-term and long-term, but the applicants focused on short-term credit to avoid over-indebtedness (both personal and institutional) and collateral security. PDF observed that the credit limits set for fixed investments are too low.
13/ The project called for setting up a credit fund administered by PDF and channelled by IFIs to members of the target group to finance the following: (i) farming operations; (ii) small rural enterprises; (iii) projects by women's and youth groups.
15/ Demand for credit from PDF presented in the following manner: (i) publicity by IFIs having benefited from PDF credits; (ii) promotional work by PDF and official institutions; (iii) requests for credit assistance by local campesino organizations; (iv) requests from TASU. In most cases, organizations meeting the criteria to operate as IFIs went directly to PDF, but with no commitment with respect to TASU.
18/ Crop diversification, replacing areas given over almost entirely to cotton with crops using fewer chemicals, and avoiding exposure of larger areas to the pressures of indiscriminate mechanized soybean growing, have contributed to preserving natural resources and the environment. Also, the introduction of practices to conserve and improve soils have made a real contribution to reducing adverse factors and realizing the potential of natural resources.
19/ The effects of initiatives in this area were not immediately clear. It must be remembered that only in 2002 did TASU hire a specialist to ensure that all project actions built in a gender focus, although a diagnostic assessment of the gender situation in the project area conducted with support from IFAD's PROGÉNERO programme had pointed out deficiencies in 2000, and TASU had been operating for three years.
24/ The supervision reports are a valuable help in project monitoring and decision-making by IFAD. Accordingly, regularity in submission and shorter lags between the mission and delivery of the report should be made a requirement.
26/ Work, to motivate and train management and members of the IFIs included in the project, began late. As a result, minimal management and administrative knowledge was transferred to IFI members by the project, and most technical assistance for producers had little, though positive, impact.
Integrated Agricultural and Rural Development Project in the Governorate of Siliana (2004)
Interim evaluation 1
The overall objective of the Integrated Agricultural Development Project in the Governorate of Siliana (PDARI), initiated in 1997 as part of the national and regional strategy of soil and water conservation and rural poverty reduction, was to realize the production and revenue potential of small agricultural holdings as well as to improve basic infrastructure available to poor populations, in 70% of the territory within the Governorate of Siliana. To meet these challenges, the project called for investment components involving working capital, natural resources susceptible to erosion and basic infrastructure, as well as components to create and strengthen capacities among development institutions and smallholders. Major project components addressed soil and water conservation (28 000 ha), forestry and pasture development (pasture plantings on private and collective land and agroforestry activities), agricultural development of 10 000 smallholdings (promotion of fruit tree cultivation and rehabilitation of irrigated perimeters), upgrading of basic infrastructure (drinking water and desert trails) and support for women's and community-based development. The target group comprised 12 000 families, including 10 000 smallholders and 2 000 young households.
The participatory approach proposed for this project was innovative for the Governorate of Siliana, based on the following:
- programming instruments (two-tier development plans) to promote greater involvement by populations in the planning process for development actions through development consultative councils for at the imada (CCDI) and délégation (CCDD) level, which were to play a central role in mobilizing populations and in programming and monitoring results.
- the territorial intervention unit (micro-zone or sub-basin) for undertaking multisector development actions for forestry and pasture development, soil and water conservation works, and productivity improvements for agriculture;
- NGOs, which were to provide small loans in the context of local savings and loan committees within a village-type credit system to be put in place;
- women's and girls' issues, provided for through specific mechanisms to promote the economic and social role of women in rural areas; and
- promotion of the participatory approach by means of 12 pilot operations for the participatory approach (POPAs) in 12 selected douars (villages) in the project area beginning in year two, working with CCDIs.
The project's history has been punctuated by a series of internal events in connection with organization of the project management unit (PMU) and external events caused by institutional changes in microlending that have influenced the project's operating instruments, the conduct of its activities, and the nature of its results. In particular, the creation of village microlending associations was eliminated when Banque de solidarité tunisienne (Tunisian Solidarity Bank - BTS) was founded in 1997. As a result, the approach that was to be applied in conjunction with the system of CCDIs, CCDDs, PADIs2 and PADDs2, so as to come up with an authentically participatory planning strategy for the project as a whole, was reduced to just the pilot experience in two micro-zones, designed to develop the local development approach and instruments. This was attributable to the chosen option of working only with a limited number of NGOs (FERT and FTDC3) rather than taking advantage of the capacities of all NGOs for participation purposes. Lastly, the mid-term evaluation performed by Centre national des études agricoles (National Centre for Agricultural Studies - CNEA) in 2000 led to changes in certain sectoral objectives, in particular an increase in the number of bench terraces, to the detriment of soft techniques.
PDRAI's main partners are the Siliana Regional Commission for Agricultural Development (CRDA), the Siliana BTS, the External Financing Directorate (DGFE), IFAD, the French Development Agency (AFD), the Arab Fund for Economic and Social Development (AFESD), the cooperating institution, and other support structures such as IRESA and NGOs. Their support for the PMU has been mixed. Generally speaking, the PMU has not benefited from technical support commensurate with the project's ambitious goals. With respect to DGFE support, despite the lack of a national monitoring and evaluation system for PDARI, several missions and field visits have taken place to assess the results achieved and provide guidance for improving implementation. The AFD's technical and methodological contribution was more intensive and consistent, particularly with respect to direct seeding research and development. IFAD support has been limited to physical and financial monitoring of its portfolio through FADES. No technical missions have been sent directly from IFAD aside from the one that took place in 2003 to assist the project in better structuring summary reporting and to begin capitalizing on experiences in community-based development. The PMU team, despite the constraints it faced (limited staff, lack of flexibility in budget programming, lack of continuity in staff given their contractual status, etc.), performed a significant amount of good work. However, the team did not manage to design and put in place a more effective operational strategy in time to better program activities, which would no doubt have achieved better results.
PDARI's objectives were and remain very relevant for a highly rural area that remains vulnerable to climate events, erosion, degradation of natural resources, poverty and rural exodus. Material achievements are quite substantial and, overall, do not present any major failings that cannot be overcome. However, activities were not as a whole designed according to the prescribed approach, i.e. a participatory integrated development approach by micro-basin, although several examples of integration can be observed in certain locations.
In the space of seven years, PDARI has shown notable results given the relatively ambitious objectives of limiting the effects of erosion and realizing the production potential of smallholders under a participatory approach. The project initiated a promising approach to research and development with national research institutions (INRAT, IRESA, etc.) on leading topics relating to innovations and changes in production systems in the area, particularly in tree cultivation (Bargou peach), direct seeding and animal feed diversification (food blocks). Project-supported activities to create and strengthen management skills among women, and particularly girls, have generated new revenue sources and an enabling environment for the emancipation of girls.
Project implementation took place under a sector programming approach with broad consultation of populations. Forums created for dialogue such as CCDIs did not fully perform their role in the participatory programming of development activities. The PMU's resources were limited since the project proposal did not build in sufficient financial and institutional flexibility for the participatory approach to develop in all sequences of local programming. The results of the EPIDEL experience in Hamam Kesra and Soualem were mitigated. Populations were fully involved in this process but were quite disappointed with the results obtained. Technical support provided in this connection, whether directly by the PMU or through the two NGOs FERT and FDTC, was not consistent enough to be effective.
Sizeable flows of financing were injected into the Siliana regional economy. As an annual average, major investments were on the order of TND 6.2 million (close to USD 5 million), 15% to 25% of which flowed into local economies in the form of payroll for various construction projects. The quality of infrastructure is generally good. However, some of this infrastructure has not been fully appropriated, especially bank terraces and certain hydraulic structures.
This injection of capital by means of investments involving socio-economic infrastructure, support for community-based organizations and the rapid development of certain revenue-generating activities have unquestionably had a positive impact on the dynamics of local economies in the various locations, and have improved the quality of life for their populations.
The most visible impact of PDARI activities in areas threatened by erosion has been twofold. It has improved the producing potential of smallholders and reduced silting in dams and hill reservoirs in the areas treated (21 000 ha), benefiting all categories of smallholders. These effects will translate into the daily lives of smallholders five to ten years from now by enhancing the agronomic potential of land based on capacity created for water retention, diversification of production with 6 000 additional hectares of olive trees and increased water reserves in dams and hill reservoirs downstream from the land treated. In terms of the farmers' assets, land values will increase as land becomes better protected and covered with valuable tree plantings.
With respect to basic infrastructure, the project has opened up access for 10 localities with 10 000 inhabitants and provided drinking water to 7 000 people in a number of disperse douars. In terms of community-based development, major activities have included agricultural training for 391 girls and have given rise to 107 microprojects in cattle, sheep and rabbit-breeding and beekeeping. Arts and crafts training has reached 336 girls, including carpet-weaving (194 girls) and other marketable products (18 girls trained in margoum or short-weave carpet-weaving, and 124 in pottery and decoration), with 70 microprojects established. On the other hand, only a few groups and just five of 42 CCDIs received full and adequate training.
PDARI has also made a significant contribution to creating two micro-zones that can lead the way in economic and social change. In the area of agricultural development, investments made in the Rohia délégation to develop 1 608 ha, benefiting 456 farmers, will transform the local landscape. These developments have strengthened Rohia's producing potential and created a micro-zone for intensification of agricultural production. The second subzone is Bargou, where a number of activities (e.g. forest and pasture development, integrated forestry management, hydro-agricultural development, strengthening for users' associations, Bargou peach promotion) have created a local dynamic for agricultural development to bring in real socio-economic change.
These are major accomplishments, and their impact on living conditions for many households is far from negligible. But structural poverty in the governorate, though slightly diminished, is still observable in the income diversification strategies adopted by all families (e.g. farming, seasonal work, temporary out-migration). From the point of view of improving sector and category targeting toward poor populations, the mission sees potential in carrying the PDARI experience through to a second stage, under a new approach that would take into account the requirements of a kind of rural development that is more open to innovation and off-farm activity, and better adapted to the workings of local economies.
Based on the achievements and the failings of the PDARI experience, the mission has formulated two types of recommendations. The first are general and have to do with the way the PDARI concept has evolved. The second concern certain components implemented by the project.
In a second stage, PDARI should operate within the framework of a development approach that can reconcile the imperative of conserving natural resources with creating conditions for diversifying monetary revenues. This framework could be participatory local development. The soil conservation and natural resource preservation aspects should be maintained through government or other financing given the serious erosion and degradation of vegetation in this area. This kind of local development should take into account a concern for reducing farmers' economic vulnerability while working to develop off-farm activities.
Also, during a new stage of the IFAD project, a different strategy ought to be deployed to support rural development centred around activities of various kinds integrated in coherent socioterritorial sub-spaces consistent with the workings of local economies and rural societies in Siliana. The essential feature here would be a diversity of adaptive strategies to diversify monetary income. Only through the interaction of agricultural and pasture development activities and off-farm activities, and by promoting proximity services for and by the rural world, can a participatory local development programme take on strategic dimensions to have more widespread impact on local development dynamics and, in the long term, enable populations to remain in their villages.
Furthermore, the PDARI experience has clearly shown the lack of synergy among the partners involved in implementing project activities (Government, AFD, IFAD and FADES), to the detriment of the project. In the future, and given the Government's search for new formulas for liaison and exchange with rural and farming communities (GDAP, GD, etc.), it would be advisable to test and adjust incrementally the institutional arrangements established by the project (CCDIs, CCDDs, etc.) to ensure that they are working in coordination and synergy with existing ones (CLDs, CRDs). It is also recommended that a permanent mechanism be set up for technical review of project components, and that a review of the effectiveness of steering mechanisms take place every two years so that any technical and financial adjustments can be made, and to capitalize on good practices in development.
The second major recommendation has to do with consolidating PDARI achievements, by means of the following in particular:
- enhancing soil and water conservation and forest and pasture development activities by strengthening the organizational skills of populations and their ability to take charge of infrastructure upon completion;
- consolidating research and development achievements in the context of a development vision;
- establishing a more effective monitoring and evaluation system; and
- capitalizing on instruments and approaches from the PDARI experience in the form of technical manuals and reference materials for use by CRDA and other development agencies.
1/ The interim evaluation mission was composed of: Mr Moncef Kouidhi, economist and Head of Mission; Mr Salah Rouchiche, agroforestry specialist; Mr Luca Fè d'Ostiani, sociologist; and Mr Slah Nasri, soil and water conservation specialist. Contributions were made by FERT and AGER consultants, Mr Jean-Charles Derongs, Mr Guillaume Dherissard and Ms Mouna Mastouri.
2/ Imada annual development Plans (PADIs); délégation annual development programmes (PADDs).
3/ Fédération tunisienne pour le développement communautaire (Tunisian Federation for Community-Based Development - FTDC).
Special Country Programme, Phase II (SCP II) (2005)
The aim of the Special Country Programme (SCP) Phase II, building on that of Phase I which was conducted in Oromia and Southern Nations, Nationalities and People's Region (SNNPR) between 1987 and 1996, is to "improve food security and incomes amongst poor rural households by enhancing their resilience to drought, through intensification, diversification and commercialisation of smallholder agriculture." SCP II has operated in Tigray, Oromia, Amhara, and SNNPR since 1999. The objectives of the project are to improve and expand traditional small-scale irrigation schemes, enhance agricultural support services, and strengthen the government and community institutions responsible for project implementation. The project is complex, in terms of its aims, components and institutional arrangements. The natural environment and socio-economic and political contexts are very challenging, and this must be taken into account in the assessment of project performance and impacts, as well as the formulation of a future phase of the project.
This interim evaluation (IE), taking place in the final full year of the project, was composed of a pre-mission socio-economic survey carried out in depth in three irrigation schemes and their adjacent communities, together with a four-week mission by four independent consultants, in which 22 irrigation schemes, 14 woredas and all four project regions were visited. The methodology of the evaluation combined surveys of individual farmers (mainly gathering quantitative data) with semi-structured interviews with farmers and farmer groups, woreda officials, and regional and federal personnel, and observations on site. The IFAD Methodological Framework for Evaluation (MFE) is adopted. The latter includes rating for project performance, impact and performance of project partners (provided in the main text). The evaluation mission was preceded by a survey, comprising both qualitative (focus group) and quantitative (randomised administration of questionnaires) techniques. The evaluation timing was not ideal, falling as it did in the rainy season and around the Meskel public holiday.
Main design features
Project components. The project set out to improve irrigation infrastructure on 800ha of traditional irrigation schemes, and expand command areas by a further 4 500ha. Dry season irrigation of a range of vegetable crops, the majority of which were to be sold into local markets, was to provide the justification for the investment of 70% of the total project funds. The agriculture component would consist of enhanced extension services, soil conservation measures in the catchments where the irrigation schemes are located, development of seed multiplication activities, and promotion of vegetable plots for women, in or near to irrigation command areas. A range of institutional strengthening measures would be implemented through technical assistance, training and provision of basic resources such as vehicles and equipment.
The target groups of the project would include 23 400 households farming approximately 0.25ha each in small-scale irrigation schemes; 10 000 farmers on rainfed land who would benefit from soil conservation measures; 2 400 women vegetable gardeners each cultivating a 200m2 plot; the community level, woreda, Regional and Federal institutions which would be supported directly or indirectly; and, supposedly, 2 million people in the wider community who would benefit from the availability of vegetables in local markets. The last figure lacks credibility since the population involved is composed of very poor households in the most food-insecure woredas of Ethiopia whose purchasing power is extremely limited.
Implementing partners and funding. The project is coordinated by a Project Coordination Unit (PCU) within the Federal Ministry of Water Resources, which liaises with the relevant organs of Regional Government in the four regions of Tigray, Amhara, Oromia, and SNNPR. Funding amounting to USD32.4m (including physical and price contingencies) has been made available by a loan from IFAD (USD 22.5m), GoE (USD 6.2m), farming communities (USD 3.1m, in kind), and an Irish Government grant (USD 1.34m). Supervision is provided by UNOPS from its base in Nairobi.
Changes at Mid Term Review (MTR). The main formal changes to the project design appeared at MTR in 2002. The MTR envisaged a final total of 40 Small-scale Irrigation (SSI) schemes, covering 5 190ha and benefiting approximately 21 000 households, being completed by end of project. The MTR also introduced limited provisions for the rehabilitation of schemes developed under SCP I. A new component of Water Management was added, which absorbed the activity of Water Users Associations (WUAs) establishment and added several new activities. The agricultural component was significantly restructured, with a more detailed breakdown of activities. The MTR proposed several changes to the project period, targets and costs. The MTR noted that the project had not commenced until February 1999, and so the six-year implementation period envisaged at Appraisal would run until February 2005. The MTR recommended completion at the end of the 2004-05 fiscal year, i.e. 6th July 2005, and a project closing date of 31st December 2005.
Summary implementation results
Monitoring of project achievements has been very poor. This evaluation concludes that, due to the weak M&E system, neither IFAD, nor UNOPS, nor the PCU, nor GoE more widely has more than a very general impression (from disbursements and expenditures) of the achievements to date of this project. Probably the most reliable information concerns the number of irrigation schemes completed (46 out of 58 by September 2004). There appear to have been very limited achievements in the agriculture component of the project, the most alarming single area being that of soil conservation. Achievements in relation to institutional strengthening have been mixed.
Expenditure. Overall, to July 2004, 55% of the IFAD funds had been spent, the level of expenditure across the regions varying from 30 to 60%. Only 7% of the Irish Government grant had been spent, but the main reason for this appeared to be due to misunderstandings in the Ministry of Water Resources.
Performance of the project
Relevance. The project is very relevant to its target communities, to national policies and to IFAD principles and strategic thrusts. However, little analysis appears to exist of the traditional irrigation schemes which SCP II sets out to improve. It may be that lower (capital) input options could effectively and efficiently spread project benefits to a wider target group than under the present project design. Also, while a key element of the project is the commercialisation of irrigated farming, the challenges of linking farmers in remote areas to markets which can absorb their production are very great. Difficulties of physical access, the domination of the market by traders who dictate price to farmers who have no bargaining power, and the small overall size of the market, are major hindrances to the achievement of the goal of commercialisation.
Effectiveness. In a project of this type, it is very early to be attempting to measure impact and effectiveness. In relation to irrigation development, the peak year of construction will turn out to be 2003/04, with the expectation of full realisation of benefits by farmers only after a minimum of six years (according to the assumptions at Appraisal, which tend to be optimistic) from then. The limited expenditure and achievements in the agriculture component have already been mentioned.
Targeting. The project has been well targeted at woreda level, with 70% of irrigation schemes being located in woredas defined by an international consensus to be highly or very highly vulnerable. The evaluation found that significant efforts had been made to reach vulnerable areas. Some landless individuals and households may benefit from the project through providing labour, usually through share-cropping arrangements in the developed irrigation schemes.
Efficiency. The unit (per ha) costs of small-scale irrigation development in SCP II lie between USD1 100 and 6 500 (EB9 800 and 56 700), depending on whether only direct construction costs, or full project costs, or something in-between, are included. These are commensurate with norms for this type of infrastructure in sub-Saharan Africa. It is too early to observe the full benefits, but a critique of the benefits assumed at appraisal shows that many of these assumptions (yields, percentage of crop sold, prices, need for maintenance) were optimistic. Economic rate of return is almost certainly substantially less than proposed at appraisal (15%), but it is difficult to identify interventions with demonstrably higher rates of return, as well as attractiveness to farmers and Government stakeholders.
Rural poverty impact
Physical and financial assets of target irrigation farmers have started to improve. There are instances in which ‘modernisation' of irrigation has made matters worse for farmers, but overall these are few in number. Impact on financial assets has been limited by low producer prices when middlemen are involved, poor roads, gluts, and consequently farmers' preference to retain for consumption crops which otherwise might be sold.
Impact on human assets, in the form of skills and knowledge, has been limited by the generally poor quality of extension work, unimaginative use of trials and demonstrations, and limited institutional support provided so far by the project. Concerning the impact on social capital, the establishment, strengthening and empowerment of local organisations for water management has generated confusion. Traditional water user groups have not been exploited effectively in the move toward ‘modern' organisational forms (WUAs and cooperatives). The stakeholder mandated to strengthen WUAs is focused on the promotion of cooperatives, which are unattractive to some (perhaps many) farmers because of associations with the former Government (the Derg). The situation remains confused, and a resolution to this issue is urgently needed.
Food security, in the sense of increased and more reliable production and increased income, is improving, for irrigation farmers.2
The range of dietary intake is also widening due to crop diversification. The cash generated from selling vegetables and other produce is commonly used to buy food to cover the household food demand during the food deficit months. Some farmers spoke of a reduction in hungry months from about six to two (July and August).
A positive impact on the environment, through soil conservation, is crucial to the sustainability of the physical irrigation assets, as well as to the wider spread of benefits beyond the command areas. So far very little has been achieved in this respect, although funds are available for this purpose. There is a danger of soil degradation within the command areas, if soil nutrients are not managed carefully through inclusion of legumes within crop rotation and use of fertiliser and manure. Waterlogging and salinity risks need to be regularly assessed.
The institutional context – especially government re-organisations and decentralisation – has limited the impacts of the project. Coordination among the various federal and regional stakeholders is weak and participation in planning at the woreda level is not encouraged. Conversely the project has had little demonstrable impact on institutions (in the sense of ‘rules of the game'), policies and regulatory framework. The potential for policy dialogue between donor, project stakeholders and higher levels of government is significant, and initiatives need to be taken to engage in such dialogue.
The project's impact on women through the promotion of women's vegetable gardens has been small in numerical terms (only 3.2% of available funds spent), but significant in terms of depth. This is an area of real and demonstrable importance to family life, nutrition, and women's empowerment, and should be given the importance it merits.
Sustainability of SCP II irrigation schemes depends particularly on careful site selection (especially in relation to markets and other irrigation abstractions); proper attention to social structures; respect and recognition of indigenous knowledge in the study and design process; and formal recognition of the need for post-construction support to irrigation communities. There are weaknesses in all these areas, which threaten sustainability. Two important physical threats to the irrigation schemes concern the damage caused by catchment soil erosion, and limited dry season water resources. These both highlight the importance of an integrated approach to catchment planning and management which takes account of all relevant land and water uses and users.
The project is innovative in its particular combination of components and target farmers, but its replication in the same form will remain dependent on donor funds for the foreseeable future. Some non-target farmers have been observed to imitate the technology which they see, but mostly using local materials in what amount to contemporary ‘traditional' schemes.
In many cases visited in this evaluation we have observed competition and conflict between upstream and downstream water users. At least one region takes a whole-catchment approach to the study and design of potential new schemes, cataloguing existing abstractions and computing a water balance for the site under study. This should be standard practice, and indeed Ethiopian law provides for the "Supervising body" to issue permits for water abstractions (not for traditional irrigation schemes, but presumably for ‘modern' schemes of the type included in SCP II).
The sense of ownership of irrigation schemes by the regions is an important plus-point in regard to sustained impact. The fact that the regions take responsibility for post-construction maintenance and repair, when this lies beyond the capacity of the farmers, is a recognition of reality, and an important contributor to sustainability.
Performance of partners
Partner performance. IFAD has been strong in terms of direct support to GoE, but weak in the management of supervision and technical assistance.
Opportunities to initiate policy dialogue and donor coordination in some key areas relevant to SCP have not been exploited, although it is not too late to initiate this. 3
The Cooperating Institution (UNOPS) has performed conscientiously within severe time constraints. The supervision process however has major shortcomings. GoE has demonstrated strong commitment at the level of policy; less delivery in terms of ensuring adequate human resources at all times; poor monitoring of activities; and weak coordination among government stakeholders.
As would be expected in a project as complex as SCP II, the overall assessment is mixed, with some areas of significant strength and others of weaknesses which are great enough to undermine sustained impact. Apart from those aspects already mentioned, the evaluation would particularly highlight as strengths of the project the high degree of commitment and the positive attitudes of many individuals at the level of the PCU, the Regional Programme Coordination Unit (RPCU), within the various regional government agencies involved, at woreda level, and among the many farming communities involved. Significant weaknesses exist in the processes of disbursement of funds; in the degree of joint (consultant-GoE) participation possible within the processes of appraisal, MTR, technical assistance, supervision and evaluation; and in the priority given to non-engineering aspects of the project. There are also several aspects of the original project design (especially its optimism about sustainability, crop yields, and the possibilities of commercialisation) which the present evaluation challenges.
Insights and recommendations
Part 1 - Extension of SCP II. Expenditure of available funds, especially from the Irish grant and in the agriculture component, has been limited. Monitoring of project achievements has been weak. The project is due to be completed in July 2005, with a closing date of 31st December 2005. It is unlikely that full expenditure and complete and accurate reporting can realistically take place by these dates. An extension of these dates by at least 12 months is recommended. If this is accepted, then a number of issues can be addressed even before taking any decision on future interventions.
Participative processes for the formulation of next phase. We recommend that planning begin as a matter of urgency, to define a third phase of the SCP, in full consultation with those responsible for implementing Phase II. The general and more detailed recommendations which follow Table 14 relate to the content of the formulation activity. We urge that every attempt should be made in future project formulation to make project design a fully participative process. The knowledge and experience which exists at federal and regional level of all aspects of project implementation should be utilised to the fullest extent possible. We recommend that a joint team of SCP II personnel, Ethiopian consultants, international consultants and IFAD personnel be assembled for the purpose. It is likely that formulation in this fashion will take longer than under present procedures, but the benefits in terms of realism of design and ownership of the outcome will be significant.
Formulation – producing limited but necessary paperwork. Large quantities of detailed prescriptive documentation of variable quality and usefulness, simply gather dust and fails to fulfil a useful function. We recommend the production of the minimum amount of paperwork, in formats which are agreed by all stakeholders to be necessary and useful for project management at various levels. Concise formats such as logframes which fulfil multiple necessary functions are to be encouraged.
Formulation – produce a flexible project design and recognise the need for long-term programming. The project design should be sufficiently flexible to allow variation in approach from region to region, and evolution of approach over time, as better procedures are learnt by those implementing the project. Any future project addressing food security in Ethiopia through a package of small-scale irrigation and agricultural support components should recognise the long-term nature of such an intervention. The full adoption of the project by government, and the full realisation of the benefits by target groups of farmers may take as 10-12 years or more. Continuity of effort is needed to achieve expected outcomes.
Formulation – producing procedures of sector-wide applicability. Regions and woredas in food-insecure parts of Ethiopia have their own, and other donor-supported, programmes of assistance to small-scale irrigation. Any future project focusing on SSI should endeavour to the greatest extent possible to integrate with regional, woreda and donor programmes, in order to simplify and strengthen programmes in this sector and move toward a sector wide approach. Approaches vary, but the common goal of household and national food security is shared. We recommend a joint donor-stakeholder forum to share experiences across donor programmes, broaden the menu of options with a view to the possible development of a common approach within the sector.
Part 2 - Insights – summary of conclusions from the Main Report. Table ES1 sets out in summary form the main points made at the relevant places in the IE (2004) Main Report.
Table ES1 Interim Evaluation (2004) Insights
| Project achievements are not adequately monitored. Identifying and quantifying the project achievements has proved extremely difficult. None of the stakeholders has accurate overall knowledge of project achievements to date. This information is needed for effective project management by regions, PCU and IFAD. |
| Limited achievements in the agriculture component. The agriculture component of the project is severely under-spent. The limited activities in soil conservation, women's vegetable gardens, seed production, and extension services threaten to undermine impact on key target groups. |
| Little is known about traditional irrigation systems. It appears that many assumptions have been made about the weaknesses of traditional irrigation systems, without the foundation of detailed investigation and diagnosis. It may be that less capital-intensive interventions to improve traditional systems could have significant benefits, potentially spreading benefits more widely. |
| Market for vegetables is limited. The assumption that there is a large accessible market for vegetables is questionable. Physical access to markets is challenging. Prices given by traders are very low. Purchasing capacity of rural populations in food-insecure woredas are extremely limited. High transaction costs limit the possibilities for exporting produce. Producer prices can be very low, unless farmers sell directly into the market (without middle-men). |
| There is a great deal of institutional learning to share. The experience gained by GoE and IFAD in SCP I and II puts both stakeholders in a very strong position to engage in dialogue over the outworking of policies in the areas of: food security, land and water management, coordination of Government agencies, cost recovery, and on-going support to communities. Little has been done in this area to date. |
| The logframe is not used. An agreed, detailed and up-to-date logical framework (logframe) is an extremely useful management tool. SCP II's logframe is weak, incomplete, and not used. |
| Long term commitment is needed. The impact of SCP II will only be fully realised if there is continuity of project activities over a minimum of 10-12 years. A six year project is too short to achieve significant impacts. It has taken until PY5 to reach a peak in irrigation scheme construction, and longer in the agriculture component. Benefits to farmers will take another 6-10 years to realise. |
| Targeting at woreda level is good. The project is well targeted at woreda level, with 70% of SCP II irrigation schemes being located in food-insecure woredas. |
| The appraisal assumptions were optimistic. Many optimistic assumptions were made in the economic analysis of the project at appraisal. In particular we highlight the high yields, high producer prices, low post-harvest losses, no water scarcity, and no maintenance costs assumed at appraisal. |
| Modern irrigation development is sometimes flawed. Not all ‘modern' irrigation development has benefitted all of the target farmers. Mistakes have been made in particular when engineers have ignored the knowledge or wishes of farmers, when hydrological assessments have been flawed, or where upstream developments have deprived schemes of water. |
| SCP II may have reduced grazing areas. SSI development, combined with area enclosures and re-afforestation, may have reduced grazing areas and livestock numbers in some cases. The impact of this on the environment, on financial assets, and on diet needs further investigation. |
| Insecurity of land tenure remains a matter of concern. Insecurity of land tenure, both within SSI schemes and outside, continues to be a widespread problem, of perception, and in reality. |
| Financial assets are increasing. Financial assets of irrigation farmers are rising, but slowly, because of the market problems raised earlier. |
| Access to credit is mixed. Access to credit by SCP II farmers is mixed. But low levels of usage of bought inputs (seed, fertiliser, pesticides, herbicides) limit the perception of this as a significant issue by farmers. |
| Extension services are of poor quality. The quality of extension work is low, and specific SCP II interventions such as demonstration plots and trial sites have limited impact. |
| Social organisation for water management needs to be resolved. Traditional water management organisations tend to be ignored in the establishment of ‘modern' WUAs and cooperatives. This threatens the viability of the modern structures, and is disempowering. The stakeholder charged with responsibility for strengthening WUAs is only interested in promoting cooperatives. WUAs do not have legal status to enable them to operate a bank account and access credit. Neither WUAs nor cooperatives fully represent the water users farming within irrigation command areas. |
|Limited impacts on women farmers are nevertheless encouraging. Where the project has facilitated home agents at woreda level, and women within irrgation schemes, the initial results have been very encouraging. Much more remains to be done in this key area of impact.|
| Attitudes to commercial farming are changing. Irrigation farmer attitudes to commercialisation of crop production appear to be changing, and some of this change is attributable to the efforts of SCP II. Whether these changes will persist in the face of marketing difficulties faced by farmers, remains to be seen. |
| Crop diversification is taking place; yields are mixed. Crop diversification within SCP II schemes is occurring, but yields are variable from scheme to scheme. Some vegetable yields are still well below those assumed at appraisal, even for year one of production. |
| Irrigation households are eating more vegetables. Significant dietary intakes of vegetables appear to be taking place. |
|Soil erosion is a major environmental threat, and SCP II could have a major impact. Soil erosion threatens the viability of both rainfed and irrigated farming. SCP II includes a significant component of soil conservation work, but very little has been achieved so far.|
| Intensive multiple cropping in irrigation command areas will lead to soil degradation. Without specific measures to manage soil fertility, such as rotation including legumes, and use of fertiliser and manure, soil nutrients will be rapidly depleted. |
| There is limited evidence of the beneficial effects of area enclosures on natural vegetation. |
| Government re-organisation and decentralisation have limited the impact of the project. At woreda level, under-staffing, under-resourcing, and rapid turnover of staff are major issues. |
| Impact of the project on women is more likely to be achieved through targeted activities such as women's vegetable gardens, than through women's membership of WUA executives. |
| Vegetable cultivation increases labour requirements significantly. |
| The achievement of sustainability depends on site selection in relation to markets; establishing or strengthening sound social structures; study and design which takes account of local knowledge; and formal recognition of the need for post-construction support. |
| The project is innovative in its combination of irrigation, soil conservation, female-focused and institutional support activities. At the present level of capital-intensity, it is not directly replicable without continuing donor support. However, non-target farmers are already copying what they see, and developing new ‘traditional' irrigation systems. |
| Downstream developments compete for water with those upstream, and already there is significant competition and sometimes conflict over limited water resources. We stress the importance of an approach based on integrated catchment planning, in order to limit and manage such conflicts. |
| The performance of IFAD and UNOPS has been limited by the shortcomings of brief foreign ‘expert' inputs which place more emphasis on outputs than on process. The system prevents effective development of partnership, inter-dependence and joint ownership. |
| The performance of Government has been mixed. Commitment at policy level has been high, while maintenance of staffing levels, monitoring and coordination have been weak |
| Performance of community social organisations has been limited by the confusion over traditional water management structures, WUAs and cooperatives. |
Part 3 – Recommendations applicable to project implementation and content
Policy dialogue. Areas in which the project's experience on the ground could make a valuable contribution to national policies and institutional frameworks include at least the following: water resource management at catchment level (including use of permits and the application of legislation such as Proclamations 92/1994 and 197/2000); adaptation of national water resource policies and legislation to regional level; marketing and price regulations (protecting farmers from unscrupulous merchants); policies on WUAs and so-called irrigation cooperatives; policies and practice relating to land title; and understandings and practices in relation to post-construction maintenance and rehabilitation. The project could take a highly constructive lead in future in facilitating debate and movement on these issues, all of which strongly affect the impact and sustainability of small-scale irrigation schemes.
Institutional arrangements. This project has experienced complex, and too frequently changing institutional arrangements. In light of the many organisational changes which have taken place, it is remarkable what the project has been able to achieve – despite, rather than because of, the location of organisational authority and the linkages and coordination between stakeholders. We now have major concerns about organisational changes in process at regional and federal level, and recommend that a careful and thorough internal review of the implications of these changes be set in train. Ways need to be found to avoid the loss of institutional learning and experience built up now over many years, and incorporate it into future project implementation.
Social organisation. The wide variety of approaches taken in this project toward traditional water management structures, WUAs and cooperatives speaks as much of the variety of perceptions of these organisations as of the site specific needs. Within the project as a whole there is a great deal of confusion, created by lack of respect for farmers' traditional structures, the ‘modern' belief in a standardised WUA, and the dogmatic promotion of cooperatives. We do not promote a single solution to this complex situation, but our recommendation is for regional and national debate and experience-sharing on the subject, and a high degree of flexibility in the solutions developed in different places and at different times.
Catchment planning and development. It is essential that any individual irrigation scheme is appreciated in the context of the entire catchment in which it lies. This is important from the point of view of water resource evaluation, of the assessment of soil and water conservation requirements and of the prevention and resolution of conflicts between user groups. At least one of the SCP II regions sets its scheme study and design (feasibility study) process in the context of a database of water developments in the entire catchment. This good practice should be extended to those which at present treat each scheme as an isolated entity.
Consolidation and component balance. In the early days of water sector infrastructure projects, it is common for more emphasis to be placed on physical construction than on supportive actions to extend impact and ensure sustainability. SCP II is no exception to this general rule. It has been more convenient for funds to be focused on construction expenditure by the regional irrigation authorities than to disburse money to other stakeholders such as the Bureaux of Agriculture and Cooperatives, and the woredas. We recommend that this imbalance be re-dressed in any third phase, with greater expenditure on agricultural support activities, soil conservation, women's gardens, and woreda level institutional support. We also recommend consideration of a higher level of expenditure on market access roads.
Financial aspects. The project can only move as fast as its cash flow. Some regions find the revolving fund ceiling very low, especially during peak construction periods. The system of settling accounts after disbursement is also found to be cumbersome, time consuming and inefficient. Individual receipts have to be collected from remote woredas and carried to the regions, from which in turn they are taken by an accountant in person to Addis Ababa after consolidation. Accountants who take receipts to regional offices for settlement go back to their centers many kilometers away to do their accounting work all over again in cases where errors are observed in filling forms. Informants in the regions feel that the IFAD system has to be improved, perhaps in line with the simpler procedures of some other donors. We recommend a detailed analysis of present procedures, with the aim of simplification. The pool system of accounting at regional and woreda level creates unnecessary difficulties. The new AfD system 4 has much to recommend it, and IFAD should explore this further.
Personnel and learning. There is significant turnover of staff at regional and woreda levels, resulting in a need for frequent staff orientations. Recognising this reality, the project should conduct orientation workshops for new staff, perhaps as often as every six months. More generally, there is great value in shared learning, such as that which took place at the August 2004 workshop in Adama (Nazaret). Such workshops should become regular annual events. Further learning at regional and national levels could be brought about by the establishment of policy fora in which key issues of policy and strategy could be discussed.
Scheme audit. Few consolidated data exist on the SCP II schemes constructed to date, and even less on the phase I schemes. All regions made an undertaking at the Adama workshop to compile profiles for their SCP II schemes. One region (Amhara) already undertakes a biennial review of a sample of schemes (non-IFAD) under its care. We recommend that all IFAD Phase I and II schemes be properly catalogued, and that these profiles be regularly updated. If such an exercise could be extended to non-IFAD schemes, to provide regional databases, this would greatly extend the baseline and monitoring data on which future decision-making rests.
Participative planning process. The criteria by which scheme locations are selected are not fully clear, and not necessarily the best to ensure impact and sustainability. We recommend the establishment of a participative process, involving all relevant Regional and woreda level stakeholders, to develop clear scheme selection criteria based on need, institutional capacity, and likely viability. Woredas appear to have little or no involvement in scheme study and design. We therefore further recommend the full involvement of woreda personnel in the study and design process, with more flexibility than at present in the balance of project components at any particular site. Farmer involvement in scheme planning and design is still limited, and so we press for the adoption of more fully participative processes of planning and design, in which all professional disciplines are trained and to which they are committed. One thrust of such approaches could focus on minor (low-cost) improvements to traditional irrigation which are able to significantly improve performance for water users.
Construction, maintenance and rehabilitation. We are aware of several cases in which initial construction budgets have been insufficient to complete schemes. In most regions, scheme repairs and maintenance (which are beyond the capacity of farmers, but are relatively minor for the irrigation authorities) are carried out routinely by the regional authorities. In other cases, true rehabilitation (major reconstruction/repair, often with social re-organisation) is carried out. In all these instances, funds may be obtained from SCP II surplus construction funds, from non-IFAD regional sources, from NGOs, or from funds specifically designated for "rehabilitation". While this situation demonstrates the commitment and the flexibility of the regional authorities, it gives rise to two areas of confusion: first, in establishing what is the true investment cost in a particular scheme; and second, in classifying as "rehabilitation" activities which really constitute minor maintenance. In particular we urge the realistic recognition by both IFAD and regional authorities that regular (annual) minor maintenance is needed, and that this should have clear and transparent planning procedures and an adequate (and increasing) budget line. Long-term support is a necessity, not an option.
Reporting, M&E, information flow and documentation. Quarterly reports from the regions are not presented in a consistent manner, and lack rigorous analysis and reflection. In order to ensure transparent information flows, the reporting structure and content require review. A number of implementation documents exist (including Appraisal, Project Implementation Manual, Operating Manual, Financial Manual, Supervision Reports, Mid-Term Review). Given the sheer volume of material, it is unsurprising if these documents are not fully internalised or extensively used for project management. This problem is exacerbated by staff turnover. We recommend that the Project Coordinator and Regional Coordinators meet to design a simple progress reporting structure, especially to fulfil the requirements for imminent end-of-project reporting.
1/ The evaluation was led by Professor Richard Carter (Water Sector Specialist, Cranfield University, UK), with Ato Ayele Gebre-Mariam (Socio-economist), Dr Kerstin Danert (Independent Water Sector Researcher and Consultant, Uganda), Dr Tilahun Amede (Consultant Agronomist, Ethiopia) and Ato Merkorewos Hiwet (Consultant, Marketing and Economics, Ethiopia). Mr Fabrizio Felloni, Lead Evaluator, designed the evaluation methodology, conducted a preparatory mission in May 2004 and accompanied the evaluation team during its first and last week.
2/ In the main text we argue that this is a narrow definition in view of more recent findings on nutrition security.
3/ On the matter of policy dialogue, IFAD has been engaged in the preparation of two recent programmes in Ethiopia: the Agricultural Marketing Improvement Programme, and the Rural Finance Intermediation Programme. These areas do not fully coincide with those referred to in this paragraph but are relevant to rural poverty alleviation.
4/ Which uses payment certificates rather than receipts.
Rural Finance and Community Initiatives Project (2005)
Interim evaluation report 1
The Interim Evaluation of the RFCIP is being conducted prior to the consideration of a second phase of the project. The evaluation mission visited The Gambia from 11 July to 7 August 2004. Methodologies utilised included field-level questionnaires, focus group discussions, interviews with key informants and case studies of the various community-based organisations and their members. For impact assessment, the mission relied on a quantitative survey and PRA exercise, carried out for the purposes of the mission by the University of The Gambia, as well as on official documents and socio-economic literature. The evaluation adopts the IFAD framework and provides ratings.2
IFAD has funded or co-funded six projects in The Gambia since 1982, of which two are ongoing: the RFCIP and the Lowlands Agricultural Development Project (LADEP). The latest Country Strategy Opportunities Paper (2003) introduced a lending envelope of US$ 15 million up to 2010 for the development of rice cultivation and the continuation of microfinance initiatives. Rural micro-finance was described as ‘possibly the most important issue for IFAD in The Gambia'. According to national statistics, the ratio of rural credit to rural GDP declined from 30-40% in the early 1980s to only 7-8% in the following decade.
The microfinance sector has demonstrated a growing professionalism in the provision of financial services in rural areas during the last decade, with government policy benefiting from the lessons that sustainability depends on mobilising savings, and avoiding subsidised donor-led credit programmes and ceilings on interest rates. Chief among autonomous microfinance institutions are the Village Savings and Credit Associations (VISACAs), which were established in 1989 and have been consistently supported by IFAD. The two other approaches are credit unions, which tend to flourish in more urbanised areas, and the group lending approach promoted by The Gambia Women's Finance Association (GAWFA), targeting poor women.
The agricultural sector suffered from the cessation of government subsidies during the Economic Recovery Program (ERP) of 1985, and also from the fall in the market price of groundnuts, the main cash crop. Current government policies aim at the revitalization of agriculture through a strategy of public and private sector partnership, with the private sector expected to assume a leading entrepreneurial role. There is special emphasis on improving food security by increasing and diversifying cereal production, and creating rural employment to help reduce disparities between rural and urban incomes.
The RFCIP was approved in December 1998, declared effective in July 1999 and scheduled for closure in December 2005. The project appraisal estimated the total project costs at USD 10.6 million, to be funded by IFAD (USD 9.2 m), the Government of The Gambia (USD 1.0 m) and the beneficiaries (USD 0.4 m in labour contribution). IFAD was the only funding donor and also responsible for project supervision. The goal of the project was the improvement of household food security and incomes in rural areas through the development of agricultural production activities and increased access to rural microfinance services, with strengthened traditional organisations (kafos) participating in the planning and implementation.
The main activities of the project under the Rural Finance component were to be the building or renovation of VISACA premises, the provision of a re-financing facility by financial NGOs, training and technical assistance at all levels and the creation of a VISACA Support Centre. Contribution of grant-equity to village community projects through the Farmer Partnership Fund (FPF) accounted for 24% of baseline costs. Agricultural support comprised participatory research, technology transfer, livestock vaccination and the building of storage facilities. Capacity building for the kafos aimed at enhancing the operation of VISACAs and promoting income-generation. Support for the growth of the VISACA movement was thus to be allied to the promotion of viable investments for farmers. This twinning of microfinance with agricultural support and capacity building constituted the basic rationale for the original project design. The heartland of the project was to be two divisions, the Lower River Division (LRD) and the Central River Division (CRD), where microfinance and agricultural activities were to be implemented side-by-side. The expansion of the VISACAs, on the other hand, was to take place in all divisions. No ‘vertical' targeting was proposed, a certain homogeneity within villages being assumed. On the basis of an eventual total of 80 VISACAs, the number of beneficiaries of the RF component was estimated at 100,000 ‘clients', including 45% women. For the kafo capacity building component, it was envisaged that 300 kafos would be involved, or around 30,000 men and women.
The Department of State for Agriculture (DOSA) was the chief implementing agency. A Project Support Unit (PSU) was set up within DOSA to handle day-to-day coordination and management of activities. Contracted NGOs were responsible for the support and training of VISACAs. The VISACA Promotion Centre (VPC) was responsible for one network as well as acting as the coordinating body. A strengthened Rural Finance Unit within the Central Bank carried out the monitoring and regulatory role. The field-level implementation of the agricultural components was carried out mainly by the Department for Agricultural Services (DAS) and the Department for Livestock Services (DLS) supervised by PSU field coordinators in each division.
Changes in policy and institutions during implementation. A Microfinance Department (MFD) was created within the Central Bank, a recognition of the key role of the microfinance sector in the country and directly linked with the expansion of the VISACA network. The physical targets in the RFCIP Rural Finance component were substantially cut by the 2002 Mid Term Review. The VPC was replaced by a new backstopping institution, the Microfinance Promotion Centre (MFPC) and the Farmer Partnership Fund (FPF) was moved to the Kafo Capacity Building component. Within the Project Support Unit, the post of M&E officer was replaced by that of Agricultural Development Officer after the resignation of the former. The stipulated positions of assistant and analyst for the Rural Finance component were never filled.3
There are now 62 VISACAs countrywide. Total membership has almost reached the target of 35,000. Savings stand at GMD 27 millions and loans at GMD 14.7 millions. Net profit (subsidies not netted out) increased rapidly during 2003 as a result of a 127% increase in interest income. However, little has been achieved in terms of the level of portfolio at risk and the capacities of committee members and cashiers. The level of capital is still low, with over 80% of equity consisting of project-funded buildings (donated equity). VISACAs increasingly seek external lines of credit since the interest rate on external lines of credit is 16%, compared with interest on term deposits of up to 25%.
VISACAs have gradually departed from the original concept of village-based institutions owned and run by the community they serve. Some now cover more than 20 villages at a distance of up to 17 km from the founding village. This has resulted in a rise in defaults and impractically large management committees because of the need for all villages to be represented. VISACAs increasingly insist on physical guarantees as a loan condition, preventing the poorest from accessing credit and reducing the focus on savings.
Support to farmers included the vaccination of around 40,000 sheep and goats and nearly 8,000 poultry and the beginnings of a programme of poultry development. Six intensive feed gardens were established to provide feed for lactating and sick animals but few farmers adopted the practice. Support for crop development comprised the establishment of 34 multipurpose gardens, the distribution of cuttings of sweet potato and cassava, improvements to marketing facilities, and training programmes for farmers in integrated pest management, soil fertility management for upland crops and the processing of cassava.
Two innovations were introduced in the agricultural sector: (i) the recruitment and training of 193 Village Auxiliaries for service as volunteer extension workers in their own communities; (ii) the introduction of a voucher-based system for the remuneration of extension officials by which their performance is monitored by village and kafo heads. RFCIP also funded an upland conservation and gully rehabilitation programme for areas affected by severe gully erosion.
The balance of the PSU changed as a result of the replacement of a dedicated M&E officer4 by an agricultural development officer and moving the FPF away from the rural finance component (as recommended by the 2001 IFAD Implementation Support Mission). The composition of the PSU is now slightly skewed against the Rural Finance component, which was to be responsible for 58% of the baseline costs of the project. Neither of the two incumbents of the post of Project Coordinator had a background in micro-finance (reportedly due to the lack of candidates with such qualification).5
Monitoring and Evaluation. International technical assistance was provided to set up the M&E system, and additional backstopping was made available. Despite these provisions, the M&E component did not achieve its primary objective: the collection and analysis of the information necessary to enable decision-makers to diagnose and resolve problems. The Participatory Monitoring and Evaluation implemented since December 2003 is far from being fully operational.
Were the objectives relevant? The three-pronged approach indicated by IFAD's 1997 COSOP–the provision of micro-credit, the diversification of crops and incomes, and the promotion of investments for farmers – constitutes a sensible response to the major problems faced by rural communities in The Gambia, and the kafos (predominantly women's kafos) were rightly seen as an appropriate entry point for project activities.
The VISACA component operated countrywide and the other components only in two divisions. Out of some 130 project sites, almost half are VISACA-only interventions and about half agricultural-only, with a mere half-dozen having both financial and non-financial activities. Since CRD, with the largest number of existing VISACAs, was excluded as a location for new ones, the main geographical focus of the agricultural components had no link with the siting of new VISACAs. The targeting of agricultural activities resulted in a more or less random coverage, often in villages where no VISACA existed. The ‘cluster' targeting envisaged at design was neglected, the synergy essential to the proper working of the project was lacking and the impact correspondingly weak.
The IE survey shows that the RFCIP failed to target the poorest households and kafos, estimating average incomes of RFCIP households in 2000 as almost double those of non-participating households. The selection of locations for new VISACAs on the basis of a certain level of economic potential and dynamism was desirable from the point of view of sustainability but also likely to exclude the poorest communities. This was exacerbated by the VISACAs' growing insistence on physical collateral as a condition for accessing loans.
Lengthy PRA and serious manpower shortage in the extension services. A large number of detailed appraisals were carried out to ascertain whether villages/kafos met the targeting criteria in terms of development levels and whether the request for project intervention reflected the needs and priorities of the communities. The appraisals involved a lengthy field work and large teams but the end result was a painstaking document of around 30 pages, much of it unutilised. There is a manpower shortage in the extension services resulting from the cut-backs during the Economic Recovery Programme and the leakage of staff to the private sector due to the lack of incentives and motivation within DOSA. The 2002 Department of Planning survey shows that 65% of villages were visited by agricultural extensionists less than once a month, and the IE survey of July 2004 found that the quality of the services received only a moderate rating from farmers.
The Farmer Partnership Fund with its cumbersome proposal and authorisation procedures now accounts for the lion's share of RFCIP time, money and effort. The concept of credit-funded community initiatives managed by the Rural Finance component has been replaced by that of grant funding managed by the Field Coordinators and the Project Coordinator. The previous ceiling of $6,000 imposed on the mini-projects has been lifted. The FPF has become a development fund awarded to whichever communities can come up with a well-argued proposal meeting the eligibility criteria.
Lack of measurable impact. The Appraisal ERR estimate of 23% was based on exaggerated incremental effects on upland crop expansion and net income. There is no evidence for increases of the order envisaged and the IE survey reveals a notable lack of impact in areas where it might have been expected. An approximate figure for cost per beneficiary is USD 80 and on this basis RFCIP was not a costly project. The problem is that many of the activities constituted one-off interventions in widely scattered communities, with scarcely discernible impact on incomes.
The operating self-sufficiency of VISACAs was only 65% if subsidization is taken into account, a very low level considering that many VISACAs have been operating for a number of years. There has been too much emphasis on the physical structure and appearance of buildings: the average cost of a new building is around USD 10,000 at current rates. It appears that the requirement for local labour is often ignored, which reinforces the sense that the new buildings are government gifts.
Consultancies replacing M&E. The gradual demise of the M&E system meant that impact assessment and diagnostic studies which ought to have been carried out as part of the routine responsibilities of project M&E were contracted out as ad hoc consultancies. The cost of a consultancy on cereal banks was USD 7,400 and the findings were such as a functioning M&E system would have discovered without outside assistance.
VISACAs have provided access to financial services, putting an end to their members' dependence on shopkeepers and moneylenders. VISACAs were easily the primary source of loans for survey respondents – 61% compared to 5% for commercial banks. Savings among VISACA households have risen by 66% in three years, but two-thirds of savings are kept in formal banks, indicating that VISACAs are perceived as better for credit than for savings. The assets of VISACA households' enterprises have risen between 2000 and 2003 contrasted with a fall for non-VISACA households. This seems to indicate that VISACA households have been able to ‘protect' their enterprise assets by accessing loans when necessary.
Limited effect on income levels. A substantially higher increase in nominal average incomes for non-VISACA clients constitutes one of the most disappointing findings of the IE survey, suggesting that participation in the project did not entail percentage income increases. Data from the 2002 internal evaluation conducted by the Department of Planning of DOSA and anecdotal evidence suggest that significant income increases existed only in those (few) areas where VISACA and agricultural interventions were combined but such increases were much lower when a representative sample of all households and intervention types was considered.
Positive effects on school enrolment and food security. A number of women kafo members reported that they were now able to pay school fees with the income from their gardens. The IE Survey indicated that VISACA loans were utilised by nearly one-third of respondents for the payment of school fees. Five out of the seven project villages covered by the IE PRA exercises said that food security had improved because of the use of loans to buy foodstuffs or through increased rice production made possible by loans, or through consumption of produce from the new vegetable gardens. The building of cereal banks has reduced crop wastage and provided protection for stored grains against rodents and birds. The revival of findo6 cultivation from its near demise has also been an important initiative given the resilience of the plant.
Environmental issues. The problem of soil erosion and local flooding has been tackled at a number of sites through the construction of bunds and Irish crossings. At the sites visited by the mission, the bunds are far too low and the whole system (which involved several kilometres of digging) will need refashioning within a few years. Grasses have been allowed to grow in the channels. The villagers appear to have no real sense of ownership of the schemes. The improper use of synthetic pesticides has been addressed by the project using an Integrated Pest Management approach, which involved the training of kafo members in the preparation and the use of botanical insecticides such as neem and papaya. Farmers can now prepare their own formulations and apply them in the field at little or no cost.
VISACAs as Institutions
VISACA financial products. With the same interest rate applied to all term deposits, there are no incentives to members to save for longer periods, and the average amount of outstanding term deposits is substantially lower than the average amount of outstanding loans. Only access to lines of credit has enabled most VISACAs to manage their liquidity ratio, and some still have ratios well below the minimum 15% required by the Central Bank. VISACAs offer short-term loans for productive and ‘non-productive' purposes. The average loan duration in 2003 was five months. Despite the need for longer-term loans for investments, VISACAs can not offer medium term loans to their members. This is due to the short-term nature of the deposit structure and to the VISACAs' inability to manage a medium term loan product.
External credit and deteriorating quality of loan portfolio. The lower interest rates charged on lines of credit (15 to 18%), contrasted with the high interest rates on deposits (up to 25%), cause many VISACAs to seek external loans and this trend may intensify, thus increasing their dependency on external ‘cold money'. Most VISACAs, especially those with insufficient deposits, would not survive a cessation of the external credit supply. Although the recovery rate to NBFIs is said to be around 95%, the trend is downward. Some NBFIs are considering withdrawing from the programme because of high recovery costs. Interest charged on loans is mostly between 30 and 36%. The question is whether the current real interest rate applied by VISACAs will be sustainable when project subsidies cease. At the end of 2003, 11% of all loans were doubtful, up from 3% in 2001. The increasing amount of default loans over the years is eroding the sustainability of many VISACAs. Faulty loan application appraisal, extended area coverage without adequate means for loan recovery and favoritism in loan allocation are held responsible.
Weaknesses of VISACA governance and lack of incentives at all levels. (i) In VISACAs serving many villages, some villages may not represented on the committee; (ii) there is very little turnover of members; (iii) women are represented in most cases but rarely participate fully; (iv) regulations regarding the functioning of the VISACAs are either lacking, incomplete or unclear; (v) there is an over-reliance on the NGO animators, who keep the books and are responsible for savings mobilization and loan recovery; (vi) the much needed adult literacy courses have not been provided. VISACA members receive no dividend; cashiers and committee members receive a small amount of money only if their VISACA shows a profit at the end of the year, which is often not the case. Recruitment of well trained cashiers is therefore problematic.
Impact of the RFCIP on the microfinance sector. Due largely to the expansion of the VISACA network, the Rural Finance Unit of the Central Bank was promoted into a fully-fledged Microfinance Department. The Central Bank has set up a regulatory framework for banks and non-bank financial institutions, including VISACAs, although the rating system tends to emphasize the number of movements and size of membership rather than financial performance, penalizing smaller VISACAs. The Microfinance Promotion Centre started operations in March 2003 as a national technical service provider, with significant support from the project. It is seriously under-staffed given its proposed role as a technical service provider for the whole microfinance sector.
Gender impact. Women account for 40% of VISACA membership and 60% of deposits, but only 27% of the loans disbursed in 2003 due to their inability to provide physical guarantees for loans. The representation of women on management committees is adequate, with some VISACAs insisting on gender parity. However, nearly all the women are illiterate and generally leave the men to lead the discussion and take the decisions. Around two-thirds of kafo members assisted by the project are women and the most dynamic of the vegetable gardens visited by the mission were run by women. The decision to disqualify milling machines from the menu of FPF is contentious in view of IFAD's strong commitment to reducing the workload of women. Women constitute around 20% of VAs and should perhaps be preferred since married women are unlikely to move away from their villages.
Only VISACAs with good credit discipline and strong leadership are likely to be sustainable. With an average 65% operational self-sufficiency for the whole network, most VISACAs are unsustainable even in the short-term. The reasons for this can be summarised as follows: the high level of portfolio at risk; the low level of savings and loans portfolio; the lack of diversified business opportunities within the area of coverage; insufficient profit to cover the interest payable and operating expenses; the failure of NGOs to deliver the necessary transfer of skills and knowledge. No exit strategy was planned in the early stages of implementation, and none has been included in NGO work plans. VISACAs are still not asked to contribute to operating expenses such as training and animators' fees. The process of institutionalisation has been promoted by RFCIP and NGOs, but the setting up of network apex bodies is in its early stages.
Urgent need for incentives in extension services. The extension service is subject to an acute shortage of manpower and the VAs are no replacement for highly trained extension officers. The only sustainable approach is to put in place attractive packages for extension staff, including regular training programmes, local and overseas study tours, part-time or short-term university courses, instruction in IT, improved office amenities such as reliable telephone connections and power supply, and performance-related bonuses. The supply of vaccines at a subsidized rate needs to be reconsidered after project closure and there will need to be a revision of current fee scales.
Outreach of VISACAs. The real innovation of VISACAs would have been to reach the poorest members of the communities they serve by developing financial products adapted to their needs, but there is no evidence for this. An "innovative" aspect of the VISACAs has been their ability to achieve a wide area coverage with a low level of implementation. Even this innovation has not been entirely positive because of the problems (loan recovery) involved in serving a number of scattered villages.
Village auxiliaries and the voucher system. The recruitment and training of volunteer extensionists represents an important innovation and the ‘auxiliary' system could be cheaply replicated throughout the country. Apart from cost, the advantages are obvious: the VAs live and work in the village in question, they speak the same language, share the same problems and are readily available for consultation. Material incentives for VAs will be required if the system is to be a permanent feature of rural life. The voucher-based system introduced to guarantee the work of extension is also an innovation in The Gambia. It minimises false claims by extension workers, encourages the coverage of remote areas and involves beneficiaries in the evaluation of the services. However, the system may serve as a disincentive for energetic officers due to the limit imposed on night-stay claims. The overall project impact was rated as modest (see main report and Appendix III).
IFAD: shortcomings of design. (i) The desired synergy between the three project components was a non-starter because of faulty targeting mechanisms; (ii) only the VISACA approach to rural finance was considered (in spite of the existing alternatives) – this meant the exclusion of the group lending approach; (iii) there was little emphasis on innovative microfinance products for the poor and the issue of medium-term loans for investment was not given sufficient consideration; (iv) no thorough evaluation of potential NGO partners was prescribed; (v) the purpose of the FPF was not adequately clarified; (vi) estimated incremental outputs and beneficiary numbers were much exaggerated.
Direct supervision. The high turnover of IFAD CPMs during the implementation period entailed a lack of continuity. Only the last two of nine missions included a microfinance expert. The problems and constraints clearly signaled at various points do not appear to have been actively pursued. It took nearly three years to replace the under-performing VPC7 and there was a volte-face regarding the structure and function of M&E, never adequately explained. The basic lack of synergy between the components was never analysed and the effects of the transformation of the FPF were missed. As for loan administration: delays in processing withdrawal applications have this year risen to 35-40 days.
The Microfinance department of the central bank has been carrying out regular supervision of the VISACA networks to ensure compliance with Central Bank rules and guidelines and other prudential ratios. Present capacities are insufficient, training levels are inadequate and the supervision schedule too tight. What will happen after the project is phased out? The Microfinance Promotion Centre has also been very active with on-site visits and quarterly reports, but it too is over-stretched.
Department of State for Agriculture. In its present state, with insufficient budgets and staffing levels, which tends to affect staff motivation, it is unsurprising that DOSA has tended to treat RFCIP as an unfocused source of funding.8 The energies of the field coordinators and animation teams have been increasingly taken up in the processing and implementation of FPF mini-projects.
The NGOs. NGO facilitators provided the backstopping for the VISACAs in return for financial support. There was no initial evaluation of the capacities of NGOs mostly concerned with grant administration in various fields. The majority of animators had no background in microfinance and some were unable to follow the basic training provided. Their functions are mostly confined to data collection and on-site book-keeping instruction, and they have been unable to provide training in planning, governance or loan appraisal. RFCIP contracts do not link the remuneration with performance; the contracts are renewable annually, but never has a contract failed to be renewed. Even in the oldest VISACAs the lack of capacity makes them dependent on animators, implying that the necessary transfer of knowledge has not taken place.
The Kafos. The prescribed training and promotional programmes have been duly carried out. However, links with the microfinance activities were tenuous. Initially, it seems that the Kafo Capacity Building component was subsumed within the Agricultural Support programme. Latterly, it has disappeared within the FPF, which accounts for the entire sum set aside for the component in the 2004 budget.
The objective of nationwide coverage by VISACAs within six years was unrealistic, and this single design feature concealed a host of problems. The envisaged linkage of components was scarcely attempted and its significance was missed by the PSU and by the Supervision Missions. The ill-conceived shift in the composition of the PSU, the failure to appoint a Project Coordinator with experience of microfinance (reportedly due to the absence of candidates with such background) and to recognise the key role of M&E, together with the predominance of FPF activities in the last years of the project, may also be traced to this initial misconception.
The key Rural Finance component accounted for 57.9% of the budget at appraisal, and 36.6% after the move of the FPF. In the 2004 budget, it accounts for a mere 4.6% of the total. Yet the various reports, including this one, still treat the VISACAs as the heart of the project. The Rural Finance component has in effect been progressively marginalised. The weaknesses of the VISACAs were identified by the IFAD Implementation Support Mission in 2001, but the decisive measures necessary to resolve them were not taken. The Central Bank criteria continued to encourage VISACAs to expand rather than to retrench. Contracts with NGOs were not rewritten so as to insist on a timed exit strategy, nor were payments made conditional on performance. The MTR rightly revised the physical targets but stopped short of demanding other radical solutions.
Some agricultural activities funded by the project, such as the support for poultry farming, vegetable gardens, cassava multiplication and findo cultivation, match the available resources and comparative advantages of farming in The Gambia, but little effort has been made to address the crucial problems of processing and marketing. Simple and inexpensive processing techniques, for example for the preparation of sauces and juices, were not propagated and the need for up-to-date marketing information was neglected. DOSA field staff with relevant expertise are in short supply.
This report emphasises that the RFCIP failed to reach the poorest segments of the rural population but also admits that sustainability of village-level financial institutions requires a certain level of economic potential and capacity to save. This poses a conundrum for IFAD, with its specific brief to target the poorest of the poor. The assumption of the Appraisal Report that no ‘vertical' targeting was necessary was true only for the more remote villages and hamlets. The overall project was rated as modest.
Future IFAD investment in The Gambia requires a radical rethink. A programme of Rural Finance sectoral support, including group-lending strategies, might be more effective than direct support for VISACAs only. The VISACA ‘brand' has been established as the unique village-level financial institution in rural areas of The Gambia, but VISACAs are becoming area- rather than village-based institutions and their modus operandi is mostly based on physical guarantee rather than the amount of deposit or peer pressure. They should put more emphasis on savings mobilization and quality of loan portfolio, and there is a pressing need for greater capacity, particularly in terms of basic literacy.
Rural finance initiatives might still be successfully twinned with support for the agricultural sector provided that a more than a nominal synergy between the two components existed. Agricultural activities should include the expansion of the Village Auxiliary network and the provision of training in vegetable processing, drying, storage, crop planning and marketing.
The Farmer Partnership Fund. The PSU and the PSC should at once reconsider the budgeting and authorisation process for the FPF, which currently take up a disproportionate amount of resources. The ultimate responsibility for the FPF needs to be clarified and decisions made about its objectives. Options should be simplified, appraisals drastically abbreviated and the USD 6,000 ceiling restored.
Relocation of the management unit. In any future microfinance/agricultural project, the project management unit should be established outside the capital, for example at the NARI compound in Sapu, which was in the past and could be again a dynamic centre for rural regeneration. This relocation would serve to intensify the project's field presence while also strengthening its autonomy and identity.
Project support unit. Changes in the personnel of the PSU should be subject to the recruitment process set out in the Appraisal Report, and changes in the composition of the PSU require an explicit rationale. In a Rural Finance project, the PC should have a background in microfinance.
Monitoring and evaluation. A centralised M&E system should not be curtailed or downgraded. Participatory M&E is a useful tool to involve beneficiaries in the monitoring of impact but cannot replace a centralised monitoring system as a basis for decision-making.
Supervision. Effective supervision of the project requires continuity and appropriate expertise so that problems may be promptly diagnosed and resolved. The constant turnover at CPM level and the lack of a microfinance expert for the first seven missions seriously affected the quality of supervision.
1/ The Interim Evaluation Mission was composed of: Professor Roger Norman, Team Leader; Mr. Issa Barro, Microfinance Specialist; Mr. Sainey Keita, Agriculture Specialist; Mr. Mark Keating, Evaluation Information Officer, OE. Mr. Fabrizio Felloni, Lead Evaluator, (OE), carried out a preliminary visit in April 2004, designed the evaluation methodology and accompanied the mission from 11 July to 19 July 2004. The IE survey was carried out by Professor Sulayman Fye, University of The Gambia, in collaboration with Mr. Edrissa Ceesay, Statistician, Central Statistics Department and Mr. Musa Suso, Research Assistant, NARI together with a team of student enumerators from the University of The Gambia. Two of these enumerators, Mr. Lang Sanyang and Ms. Kaddy Jammeh, acted as interpreters for the IE mission during field trips.
2/ Summary ratings are shown in the text, details are in Annex III.
3 / On the non-appointment of the analyst and secretary for the RF component, the Mission acknowledges that no budget was provided in the Detailed Cost Tables.
4/ The ‘dedicated' M&E Officer in this instance means the officer specifically assigned to that task and no other. No comment is intended on the qualities of the previous incumbent, who was not met or interviewed by the Mission.
5/ The Mission never questioned that the correct proceedings were followed in appointing the Project Coordinators. It merely considered it unfortunate that a candidate without the appropriate background in microfinance was selected. This was, reportedly, because none of the candidates interviewed had such a background.
6/ Findo (Digitaria exilis and Digitaria iburua) is a local cereal variety.
7/ It is not suggested that the dismantling of the VISACA Promotion Centre was an easy task, but the Supervision must share in the responsibility for the long delay before any action was taken.
8/ The Mission found that the distinctions between RFCIP and non-RFCIP activities (in extension for example) were not clearly made in the districts where RFCIP was the major source of funding. This does not imply a misuse of funds.
District Development Support Programme
Completion evaluation 1
The completion evaluation of the DDSP was converted from an interim to a completion evaluation mainly due to the realignment of the policy of the Government of Uganda towards a sectoral funding approach, which appeared to make a second phase unlikely. The task of the Completion Evaluation mission has been to evaluate the DDSP and to make recommendations concerning IFAD's future strategy and approach in Uganda, with a second phase or follow-up programme as possible options among others. The mission was in Uganda from November 19 to December 10 2004, meeting with stakeholders in the five programme districts and at central level.
IFAD has funded or jointly funded 13 projects in Uganda since 1981, shifting in the 1990s from area-based interventions focusing on agricultural development to multi-component rural development projects co-financed by the Belgium Survival Fund (the DICDPs) and to projects concerned with export products such as cotton and vegetable oil. The three most recent projects constitute a further shift, towards countrywide sectoral approaches. The DDSP was conceived as a follow-up to the rural development projects in the western region and designed to strengthen the process of decentralisation which had started in Uganda in the late 1980s.
The evaluation follows the IFAD methodological framework for Programme Evaluation, which outlines the main evaluation criteria and provides a rating system. The methodology utilized by the mission for impact assessment included: (i) semi-structured interviews with groups and committees; (ii) focus group discussions; (iii) one-to-one interviews with householders and key informants; and (iv) the review of official documents and socio-economic literature. Poverty ranking exercises were also carried out with members of community-based organisations.
Main design features
The DDSP is a USD 21.5 million programme co-financed by the Belgium Survival Fund and Development Cooperation Ireland 2 and directly supervised by IFAD. It was declared effective in May 2000. The initial closing date was December 2004, later extended to June 2006. The target area comprised the three western districts of Hoima, Kibaale and Kabarole, with a combined area of 18 000 sq. km. and a population of 1.4 million. These districts were among the poorest in the country, with average earnings less than half the national average and malnutrition still widespread.
The main emphasis of the DDSP was to be on productive activities combined with consolidation of the previous socio-economic interventions. The direct beneficiaries were to be around 50 000 households, with a particular emphasis on women and on children at risk, and a broader focus on poor farming households, fishermen and traders. The associations and committees formed under the DICDPs were to provide the basis for community participation.
The goal of the programme was to alleviate the chronic poverty in the programme area through raising rural incomes, improving health, food security and nutrition, the participation of farmers in commercialised agriculture and the enhancement of local governance. For Kabarole, which had not benefited from community-based development assistance, the programme was also designed to create the necessary institutional frameworks. The programme components were: community development, rural finance, health and nutrition, water and sanitation, rural roads, agricultural development, and management capacity building and institutional strengthening.
The Ministry of Local Government had overall responsibility for the programme and for providing the liaison and secretariat capability through a dedicated unit at central level. To the Inter-Ministerial Policy Committee was given the advisory role on policy and strategy issues. The implementing agencies at field level were the district administrations, with a District Technical Planning Committee acting as the key coordinating body. The Uganda Women's Finance Trust was selected for the implementation of the rural finance component, including the creation and guidance of groups and the management of the credit fund.
Changes during implementation: (i) the division of the former Kabarole into three districts by the creation of the new districts of Kamwenge and Kyenjojo resulted in a one year delay in start-up in the new districts; (ii) the government decision to abolish community contribution in basic health care caused the abandonment of the drug revolving fund and militated against the DDSP approach to sustainability; (iii) the operation of the health and agricultural services was affected by the ceilings on staff recruitment during the restructuring of local government; (iv) the proposed ten-year suspension of Graduated Tax has already had a significant impact on local government revenue flows, and no replacement plan has yet been announced. Design changes during implementation include the setting up of the Bunyoro Toro Development Company (BUTO), a limited company jointly owned by the five districts, to take over the running of the rural finance component.
Summary implementation results
Community development (12% of base costs). The major activities have included the construction of community centres, the training of literacy instructors and Community Development Assistants and the promotion of adult literacy groups, agricultural commodity interest groups and women's savings and credit groups. The component has also assisted in the formation of users' committees for other components. Community centre construction has been satisfactory, and the maintenance committees function well. The main problem for the component is the shortage of staff (in some districts) and dependence on unpaid volunteers.
Rural finance (14%). There is now a Subcounty Integrated Development Association in every subcounty in all districts, as planned. The SIDAs have been able to mobilize over UGSh 400 million in savings and many should soon be able to operate as independent microfinance institutions, without external support. The transition to BUTO management has been smooth due to the retention of trained staff and the intensive training of SIDAs in technical and managerial aspects of savings and credit operations.
Health (8%). Health has the smallest of component budgets, with very small amounts allocated to the new districts, where primary healthcare coverage is still very limited. The major activity has been the construction of nine Level III Health Centres, three each in Hoima and Kibaale and one each elsewhere. The quality of construction is satisfactory, but basic amenities were missing from the original design, notably running water and electric lighting for the maternity units. Training courses have concentrated on village health workers, traditional birth attendants and community health committees.
Water and sanitation (21%). The major investment has been in borehole construction, which is complete in Hoima, Kibaale and Kabarole. Water source committees and hand pump mechanics have been trained for the maintenance of the boreholes. Water tanks and ventilated pit latrines have been installed in primary schools, the latter with handwashing facilities as part of the design, maintained by pupils under the guidance of school health committees. In Kibaale and Hoima, sanitation platforms were provided to households at subsidised prices. Hygiene improvement included the promotion of handwashing, washing racks and rubbish pits.
Rural roads (17%). The largest item of expenditure has been the construction of 1 120 kms of community access roads, with 87% of these concentrated in Hoima and Kibaale. The most successful approach to construction has been the direct hiring of local labour gangs. Considering the low costs and lack of skilled labour, the standards of construction are commendable and community maintenance has generally been effective. The other major investment under the roads component is the ongoing construction of two motorable roads to Lake Albert.
Agricultural development (14%). Activities included the distribution of improved crop varieties (especially for bananas and coffee) to progressive farmers for multiplication and further distribution, the setting up of beekeeping centres, the establishment of fish fry ponds and the distribution of improved livestock. Over 20 study tours to other districts were organized and training offered in animal traction, fish farming, farming as a business, and crop and animal husbandry. The hiring of agricultural management specialists has resulted in advances in market orientation and farm record keeping, especially in Kibaale and Hoima, and beginnings have been made in linking farmers with the private sector.
Capacity building and institutional strengthening (14%). District-level staff are proficient in standard computer applications and there has been extensive training on the logframe approach to monitoring. Procurement delays have been reduced to a standard two months. Subcounty and parish officials have received training in record keeping and planning, and timely workplans and budgets are now produced at parish level. Eighty-two Parish Headquarters have been constructed for use in revenue collection and as community centres but implementation has been slow.
M&E and accounting. Despite significant investments, the M&E system is not fully functional and a central repository for data is lacking. Roads, Water & Sanitation, and Education all use separate systems. However, the districts are adept at compiling financial records and component-specific data by hand. Accounts are accurately kept and detailed financial statements produced. The continuing delay in counterpart funding – USD 481 000 at the end of August 2004 – is disrupting accounting procedures and causing uncertainties in payment schedules.
Performance of the programme
There have been significant improvements in capacities, service provision and infrastructure, especially in the older districts where the bulk of the investment has been made. Safe water supplies, the road network and primary health care have been substantially extended and the cost of infrastructure has been kept within reasonable limits. The principle of community participation in development planning has been established. Overall, the DDSP has been a successful programme, with problems related to design rather than implementation.
Imprecise objectives. The programme's stated objectives are general statements of intent rather than measurable and time-oriented objectives, and this has entailed a long search for suitable impact indicators. The ‘participation by farmers in the monetization of the economy' seems especially imprecise and does not target the poorest farmers.
Failure to target the poorest. Only the physical infrastructure components have generated benefits for households in all categories. The mission's wealth ranking exercises indicate that SIDA and farmers' group membership is dominated by the ‘not so poor'. The majority of the ‘poor' and all the ‘poorest' are unable to meet the collateral requirements of SIDA loans, which are unsuitable for agriculture. Activities under the agricultural component are mostly targeted at ‘the active poor' on the grounds that the poorest lack the basic means of production.
Decentralisation issues. The DDSP has been instrumental in promoting a process of participatory planning whereby community requests are progressively screened at higher levels. However, the long delays and lack of government response tend to generate a sense of powerlessness and frustration. There are clear limits to decentralisation, including the decisions necessarily made by technical departments, the capacities of village and parish development committees and, above all, the fact that without financial decentralisation there can be no meaningful delegation of decision making.
Users' committees. Most water sources showed evidence of neglect. Less than half of the committees interviewed had collected funds for O&M and over a quarter of beneficiaries viewed the water sources as government facilities. In many cases, there had been inadequate preparation before construction of facilities, lack of training and lack of follow up. As for community health committees, there seemed to be little clear understanding of their function and the mission assessed them as unsustainable. The situation in the road component seems better, with a widespread understanding that the responsibility for maintenance rests with the community.
Infrastructural efficiency. The cost per kilometre of roadbuilding was economical for labour-based schemes and cost-benefit ratios were very satisfactory for health centre construction. Under the water component, however, boreholes – the most expensive solution – were normally preferred even when cheaper alternatives existed. Some doubt hangs over the usefulness of the parish revenue offices in view of the imminent suspension of Graduated Tax.
High operational costs in rural finance. The Mid-Term Review allocated more for component support than for the revolving credit fund, even though most fixed assets had already been purchased. There are suggestions of extravagance in BUTO's operational budget.3 Despite the purchase of six vehicles for the component, SIDAs are obliged to hire vehicles to transport their monies.
Continuing problems with M&E. The programme M&E system has not repaid the considerable investments in terms of time and money and is still not fully utilised. Each line ministry has its own reporting requirement and corresponding software system, containing mostly quantitative data. The beneficiary evaluation process is complex and time consuming, budgets are inadequate and incentives are lacking for facilitators.
Water, roads and health. Nearly 200 000 rural people gained access to safe water as a result of DDSP interventions and the average distance to water sources was reduced to reasonable walking distance in most areas. The availability of clean water has had an impact on hygiene and in schools the health of children has improved through the provision of handwashing facilities with the new latrines. The construction of community access roads in the five districts represents an expansion of over 20% within four years. Beneficiaries reported the main impact as cheaper and quicker transportation of farm produce and better access to health centres and schools. The average distance to the nearest health unit has been reduced in all districts. In Hoima, 80% of the population now live within five kms of a health centre and in Kibaale 60%. The number of facility-based births is rising rapidly, especially in the new districts. Hoima and Kabarole have reported substantial reductions in the rate of stunting, underweight and wasting in children under five.
SIDA members. SIDA members reported a marked increase in economic activity and household incomes. Many of them have replaced thatched dwellings with iron–roofed houses and some have piped water. Others have purchased land for commercial or agricultural use and utilised the asset as collateral for obtaining larger loans from commercial institutions. Characteristics such as persistence, calculated risk taking, independence and self-confidence have been developed through training, group cohesion and access to credit for small enterprises. Over 95% of SIDA members reported running profitable enterprises.
Farmers and farming. Increases in crop yields, crop cover and livestock numbers are claimed in the agricultural sector, but convincing statistics were generally unavailable. Members of farmers' groups reported better access to health and education services and improved production skills through training and study tours. All progressive farmers could point to at least two new skills they had acquired. Farmers' groups are encouraged to save, which has in turn increased their access to credit. Farmer Extension Representatives have emerged as respected community leaders but many farmers outside the groups remain without access to extension services.
Food security. All group interviews, without exception, confirmed the perception of the beneficiaries that the overall incidence of poverty has fallen and that household food sufficiency and family incomes have improved. From the literacy classes, radio programmes and drama groups, there is increased awareness about nutrition, and many literacy students have taken up vegetable growing. Beneficiaries of agricultural programmes also reported improved diets.
Environmental impact. The environmental impact of DDSP activities has been positive, but the mission recommends a rigorous environmental impact assessment as part of the development of the lakeshore of Lake Albert after the construction of motorable link roads.4
Gender. Almost all married SIDA women said their husbands were now more respectful, and felt they were more often consulted about issues affecting their lives. A number of SIDA members have taken on leadership positions in the local administrative structure. Women have been active in the majority of users' committees.
Development impetus. The fundamental sustainability issue is whether the districts will be able to maintain the development impetus in the absence of further donor support. The problem is not with ‘hardware' aspects such as roads, boreholes and health centres but with the ‘software' aspects such as capacity building, the effectiveness and dynamism of groups, the sustainability of adult literacy programmes and the concept of agricultural commercialization.
Sustainability of SIDAs. The sustainability of SIDAs seems assured in Hoima and Kibaale, but in Kabarole, Kyenjojo and Kamwenge the associations seem weak in terms of cohesion, ownership, savings, coverage and outreach. There are some doubts about the level of SIDAs' independence from BUTO.
Volunteers. Many programme committees and activities depend on volunteers, especially under the Community Development component, which relies on the voluntary activities of literacy instructors and community ‘change agents'. SIDAs also rely on volunteers, who may not always be forthcoming. Committees under the water, health and roads components all operate on a voluntary basis and will only prove sustainable where the sense of ownership is strong.
Innovation and upscaling. The DDSP was substantially a follow-up to the DICDPs, its chief innovation being the degree and compass of direct support given to local government authorities, probably unique in Uganda. Lesser innovations include: the attempts at cross-sectoral collaboration, especially regarding the overall role of the Community Development Department; the creation by the districts of a limited company (BUTO) designed to manage credit delivery, and the involvement of NGOs in service provision.
Performance of partners
Performance of IFAD. IFAD is to be commended for the very energetic and effective supervision which in turn has led to the fostering of a direct dialogue with central government. However, the recent Thematic Evaluation complains that ‘the area of dialogue and upscaling has not been seriously explored', for which a full-time country presence is necessary.
The co-financiers. Both the Belgian Survival Fund and Development Cooperation Ireland have a significant history of assistance to the programme area and have taken a close interest in the programme. The BSF is particularly keen on cross-sectoral collaborations and is concerned that the ‘basket-funding' approach will not be able to accommodate this priority.
The Central Government. The Government of Uganda has shown a genuine commitment to decentralisation, but the ten-year suspension of Graduated-Tax without declared alternatives, the abolition of cost-sharing at health centres, the free distribution of seeds and livestock breeds in the agricultural sector, and the temporary hiring freeze in the health service have all tended to undermine the community-based and market oriented approach of the programme.
The Ministry of Local Government, and in particular the key role of its programme liaison office, is highly rated by the districts. The Ministry of Finance, Planning and Economic Development is exploring a harmonized reporting structure to be used in conjunction with the sectorwide ‘basket funding' approach.
The role of NGOs in service provision has been substantial under the programme, extending to the supply of sanitation platforms, the training of water source committees, the provision of extension services and various agricultural training programmes. Most NGOs are reported as performing well, although formal reporting procedures are not always adequate.
The Bunyoro-Toro rural development Company. It is BUTO's goal to encourage the SIDAs progressively to purchase BUTO shares and ultimately to become majority shareholders, but the feasibility of this commendable plan has not been properly examined. BUTO does not appear to operate within any supervisory framework and the reporting and monitoring systems do not conform with those of the other components. The Memorandum of Understanding according BUTO the necessary legal status for the administration of the RCF is still unsigned.
The SIDAs. The expression ‘SIDA women' conveys an image of self-confidence and entrepreneurial spirit. The Thematic Evaluation's assessment of the SIDAs was that they had had ‘a tremendous impact on women in development'. BUTO reports all SIDAs as being profitable and the longer-established ones as self-sufficient. SIDAs in Hoima and Kibaale are described as the ‘market leaders' in microfinance. The main drawback is that SIDA membership tends to exclude the very poor.
Overall. The DDSP has been a successful programme. Crucial to its success has been the demanding supervision exercised by IFAD, which has served to make good the weaknesses of the original design – precisely what supervision is supposed to do, and rarely does.
Design issues. The design of DDSP was over-idealistic in respect of ‘bottom-up planning processes'. The limits to the decentralisation of planning decisions were never properly clarified.
Programme effectiveness. The programme was effective in achieving its stated goals except in targeting the poorest households. The implementing agencies considered that for the credit and agriculture components, the ‘active poor' and the progressive farmers comprised the only practicable target groups, and that the ‘trickle-down' effect would carry the benefits to the poor. This forceful argument highlights a dilemma for IFAD, with its brief to target the poorest.
Programme efficiency. Precision about levels of efficiency is problematic in the absence of certain key statistical data. The evidence such as it is does not suggest a costly programme and indicates an efficient use of limited human resources in terms of major investments.
Overall impact. The impact of water sources, roads and health centres is undeniable, the credit component has far exceeded its measurable objectives, and the software packages (training, sensitisation and mobilisation) have had a substantial impact on ‘the culture of governance'.
Continuation of IFAD investment. The mission recommends further investment in all five districts on the grounds that the task, although well-started, is only half-done, and many of the investments may prove unsustainable. This is particularly true for Kamwenge, Kyenjojo and Kabarole, where implementation has been underway for only three and a half years. To remove a chief source of funding for these new districts5 should not be done without carefully weighing the consequences. The scale and direction of the new investments will require assessment by district and by component and the conduct of a fresh baseline survey.
Decentralisation of funds. In future projects of this kind, subcounties and parishes should be directly funded by the programme to give substance to the commitment to bottom-up planning.
Poverty focus. The rural finance component of any future project should have as a priority the development of a medium-term loan for agricultural purposes accessible to the very poor.
Strategic issues. In its negotiations with the Government of Uganda concerning future loans, IFAD should be mindful of its unique role and comparative advantages (specifically highlighted in the recent external evaluation of the Fund), neither of which are well-served by contribution to a basket-funded approach.
Counterpart funding delays. The Government should consider setting up monthly or quarterly automated disbursement schedules to avoid obstructive delays in counterpart funding.
Donor coordination. The harmonisation of reporting systems, urgently required, could be addressed at the monthly meeting of donor representatives in Kampala.
Community development. Interventions need to be aimed more accurately at the poor and marginalised. To this end, the Community Development staff should conduct training with staff of the agricultural and rural finance components on wealth ranking exercises. Men should be attracted to join literacy classes by marketing them as providing access to extension advice, English and Swahili language skills and economic opportunities.
Rural finance. Operational funding should be reduced and the revolving credit fund correspondingly increased. Clear guidelines as to the social responsibility of the component should be set out, and appropriate monitoring indicators developed. BUTO should be required to allocate a proportion of credit funds as pro-poor agricultural loans and to follow the same reporting procedures as other implementing agencies.
Health. IFAD should seek clarification on the replacement of nursing assistants before making further investments in the health service. The initial unit cost of new health centres should include necessities such as running water, DC electric power supply, a minimum number of beds and mattresses, and a perimeter fence.
Water and sanitation. As boreholes are expensive, they should be adopted only where there is no other viable alternative. For effective maintenance, communities should be trained before construction starts and a simple guide for committees should be prepared. Initiatives in some districts can be replicated: training by NGOs; associations for hand-pump mechanics; the utilisation of the O&M fund as a revolving credit fund.
Rural roads. The Ministry of Works should decentralise the responsibility for the upkeep of trunk roads to the districts, retaining only a coordinating function. It should also review its commitment to the inefficient contract system for the upkeep of district feeder roads.
Agricultural development. Tools for monitoring component indicators should be developed and relevant stakeholders trained in their use. Efforts should be made to attract poorer farmers to join farmers' groups. The policy of working with NGOs in outreach activities should be extended.
Management. The mission recommends a tolerance level of up to 10% allowing districts to deviate from the original Annual Work Programme and Budget without specific application, subject only to audit controls. Inter-sectoral collaboration would be enhanced by brief monthly meetings between component heads and inter-district collaboration by a bi-annual workshop for the exchange of ideas. The programme should hire an MS Access expert familiar with the existing M&E system to design linkage mechanisms for existing databases and to create a central repository.
1/ The composition of the mission was as follows: Roger Norman, Rural Development specialist and Team Leader; Alice Mango, Water and Sanitation specialist; Catherine Komugisha, Rural Finance and Agriculture specialist; Ismet Mustafa, Management specialist; Lea Joensen (IFAD APO) Community Development specialist. The mission was joined by Katharina Kayser, IFAD Evaluation Officer, for its final week in the country.
2 / Formerly Ireland Aid.
3/ BUTO has challenged this statement and some others in this report. See also page 38, footnote 40.
4/ Hoima District has pointed out that an environmental impact assessment was carried out for the road scheme itself.
5/ After the division of the old district of Kabarole, Kabarole itself was also effectively a ‘new' district.
Agricultural Development Project in Matam (2004)
The Agricultural Development Project in Matam (PRODAM) originated in 1989 at the request of the Government of Senegal, following the events with Mauritania and the forced displacement of border populations that followed. Carried out in 12 villages, the project was consistent with the hydroagricultural development policy for the left bank of the Senegal River.
PRODAM was launched in 1995. The project was cofinanced by a loan from IFAD for USD 16.1 million (SRS 030 SE, 98% disbursed), a loan from the West African Development Bank (BOAD) for USD 7.3 million (PR SN 92 00, also 98% disbursed), and funding contributed by the Senegalese government and the beneficiary populations for an estimated USD 4.2 million. The total cost of the project was therefore USD 27.6 million. The IFAD and BOAD loans closed in June 2001.
With a view to the preparation of a new project – which both the beneficiary populations and the Senegalese government have requested and which IFAD, considering the results achieved by the project thus far, wishes to carry out – the Office of Evaluation (OE) was asked to undertake an interim evaluation of PRODAM.
The objective of the evaluation mission was, in a spirit of partnership and joint evaluation, to assess the relevance of the project's activities and approaches, to measure the impacts and changes produced and to identify lessons for the future, but also to strengthen the capacity for analysis, self-evaluation and formulation of proposals among the farmers' organizations that participated in PRODAM in order to facilitate their future role in the design and implementation of the second phase.
Given the abundance and the quality of the studies and evaluations already undertaken on this project, a classic external interim evaluation mission was not considered justified. Consequently, the centrepiece of the evaluation process was an innovative participatory evaluation workshop, which was complemented by a desk review and a comparative anthropometric survey of the nutritional status of children and their mothers in the PRODAM area.
The evaluation process took place between January and April 2002. The evaluation team was composed of three consultants, accompanied by the responsible officer from the Office of Evaluation (OE) of IFAD.
The department of Matam, located along the Senegalese frontier with Mauritania, had a population of 215 000 in 1988. The majority of this population consisted of rural households practicing agriculture, livestock production and, less often, river fishing. Over the last three decades, a dramatic decline in rainfall (from 496 mm to 250 mm/year) and changes in the flood pattern of the Senegal River caused by the Manantali Dam had pushed the families living in this area into a situation of increasing economic insecurity and poverty. Conditions got even worse following the events of 1989 and the repatriation of 7 000 refugees, who settled in the department.
Although, as originally conceived in 1989, this project was an emergency intervention aimed at providing support for and facilitating the reintegration of the repatriated and dispossessed population in the area, it evolved into an agricultural development project whose initial activities did not get started until 1995.
Hence, in the context of the agricultural reform effort under way at the time, the overall objective of PRODAM was to enable the beneficiary farmers and herders to become highly productive agricultural operators, capable of ensuring food security for their families and improving their standard of living. The specific objectives for the project were to: (i) enhance income and food security; (ii) facilitate the resettlement of repatriates in their communities of origin ; (iii) boost agricultural productivity through the rehabilitation of village-based irrigation schemes (périmètres irrigués villageois – PIV); (iv) introduce dual cropping in the irrigated areas; (v) improve the productivity of pastoralism and rationalize the distribution of animals in the Ferlo area; (vi) maintain and improve the environment; (vii) strengthen village institutions to enable them to manage the supply of inputs needed for farming and livestock production; and (viii) initiate the process of integrating small farmers and herders into the market economy through appropriate institutional and policy initiatives.
The basis for the strategy adopted was: (i) a village approach that took account of all segments of society so as not to provoke any tensions by specifically targeting the repatriates; (ii) a focus on communities with under 2 000 inhabitants, with no known land disputes and with a sufficient amount of land in the Walo area to develop or rehabilitate irrigation schemes; (iii) sound distribution of the available land for crop farming/livestock production; (iv) a willingness to intensify existing systems of production in order to achieve better market participation; (v) a strong emphasis on involvement and ownership by beneficiaries at all levels; and (vi) contracting out, to different private or public partners, the operational implementation of the various components.
Overall, the design of the project did not change in the course of its implementation, and both the main objectives and the modus operandi remained the same.
Twelve villages benefited from the activities of PRODAM (28% of the villages in the department), with the number of direct beneficiaries estimated at 16 500 people. The principal results for the five years of project execution were as follows:
Development of irrigated agriculture. A total of 1704.5 ha of village irrigation schemes were created and equipped with high-volume power pumps. Thirteen market gardens, with a total surface area of 58 ha, were created for women's groups, and 87 km of windbreaks were planted. Only 60% of the surface area was planted in winter rice, but the average yield per hectare was over 5 tons of paddy rice. Neither off-season rice-farming nor the planned diversification activities were truly embraced by the farmers. This is explained partly by the difficulty of obtaining credit to refinance a new crop when marketing of the previous one had not yet been completed. The financial institutions involved were disinclined to take risks, owing to their past experience with delinquent loans in the area. The financial and economic viability of the land use models put in place appears rather doubtful, particularly in view of the evolution of prices for paddy rice, on the one hand, and the cost of diesel fuel and other inputs, on the other hand, in recent years. Finally, the expected gains with regard to the organization of farmers for marketing do not appear to have been achieved. Nevertheless, net household income from irrigated rice-farming has been estimated at CFAF 270 000/ha. The average total income of households in the Walo is estimated at CFAF 610 000, of which 44% comes from irrigated rice-growing.
The methodology for development of PIVs consisted in: (i) carrying out baseline socioeconomic studies to determine the number of target families and identify the land to be developed (on the basis of 1 ha per household), commissioning feasibility studies and outsourcing the heavy works to companies. The beneficiaries were responsible for the finishing-off work. Local stakeholders (pump operators, mechanics) were trained in operation and maintenance, and support was provided for the establishment of rules of use for the village-based irrigation schemes. Consolidation and redistribution of land had been organized previously. This was done in a participatory manner, with rules imposed to ensure access to irrigated land for the entire target population. Agricultural consulting services were subcontracted to the Senegal River development agency, Société nationale d'aménagement et d'exploitation des terres du Delta du fleuve Sénégal (SAED). The average cost of these irrigation schemes was USD 4 300/ha, not including support for operation and training.
Development of livestock production. The main achievements were: (i) strengthening of the operating capacity of the livestock service through institutional support; (ii) creation of three new pastoral units covering 210 000 ha and capable of supporting 95 000 new tropical livestock units (TLUs)2 (iii) construction of five cattle feed storage facilities and five vaccination pens; (iv) creation and support for the organization of three new economic interest groups (EIGs) and reorganization of five existing EIGs; (v) development of management plans for the three new pastoral units; (vi) zootechnic monitoring of 18% of the herds in the area by the Centre de Suivi Écologique [Ecological Monitoring Centre] (CSE); (vii) importation of 60 buffalo from Brazil, of which 20 buffalo were distributed on credit to farmers for animal traction; (viii) support for the Makhana farm; (ix) artificial insemination of 153 cows with Holstein or Montbéliarde semen with a view to improving milk production.
The methodology for formulation and implementation of pastoral unit management plans consisted in grouping together all villages and hamlets within a 25 km radius of the boreholes and assessing, in a mutually agreed and participatory manner, the estimated biomass resources and livestock count, and then redefining the rangeland. An assessment of the composition and production of vegetation carried out for each pastoral unit found that these areas were not degraded. Local organizations of livestock producers took responsibility for organizing livestock-grazing, fighting brushfires and preventing the unwarranted felling of trees. Through economic and zootechnic monitoring, it was possible to measure the improvement in the main parameters of livestock production, thus providing evidence of the impact of the training and advisory services provided to the EIGs. However, sales rates remained unchanged, indicating that producers did not alter their strategies in this area. The establishment of eight veterinary pharmacies helped to improve animal health, but lack of transparency in their management by the producers has weakened the functionality of these village pharmacies in recent years. The distribution of 20 buffalo among the groups made it possible to test this approach, but the results with regard to animal traction and increased productivity in village irrigation areas did not meet expectations. The artificial insemination activities did confirm the technical feasibility of this method, but its economic viability has yet to be demonstrated. The total cost for the "development of livestock production" sub-component was SDR 616 019, excluding support and infrastructure costs.
Implementation of the credit line by the Caisse nationale de credit agricole du Sénegal [National Agricultural Credit Fund of Senegal] (CNCAS). As of 30 June 2000, a total of CFAF 533 million in loans had been granted to 65 EIGs (16 173 beneficiaries), with an average repayment rate of 97%. Of the 200 loans extended during the PRODAM execution period, 98% were seasonal credits and only 2% were loans for the purchase of equipment. Overall, irrigated agriculture received 90% of the total amount of credit allocated, against 8% for livestock production and 2% for conversion activities. The objective of serving 18 250 beneficiaries through PRODAM was thus 87% achieved. In 1996, seasonal credits were granted for nine months at an interest rate of 12.50% per year, with self-financing of 20%; subsequently, in 1997, the interest rate dropped to 7.5% and the personal contribution to 10% of the loan amount.
The methodology consisted of: (i) providing institutional support to CNCAS; (ii) placing at its disposal a credit line worth SDR 1 425 000 (increased to SDR 654 000 after the mid-term review); and (iii) subcontracting this sub-component (and its objectives) to CNCAS. At the start of the project, CNCAS was unwilling to grant credit to almost all the target groups because of their past and present failure to repay loans. Hence, the project's initial efforts – which continued until June 1996 – were directed towards restructuring the EIGs and settling their accounts with the banking sector. For its part, CNCAS agreed to extend the deadlines on pre-project debts. This sub-component thus helped not only to reorganize and strengthen the management of farmers' organizations and EIGs, but also to improve their relations with the banking sector.
Promotion of credit unions (caisse populaires d'épargne et de credit – CAPECs). Results as of June 2001 included the creation of four functional inter-village CAPECs, with a total of 1 262 permanent members, 44% of whom were women. The value of the capital shares held totalled CFAF 3 858 500 (1 629 shares3), total assets of the CAPECs amounted to CFAF 2 232 149 and total deposits came to CFAF 51 million. Overall, 171 loans were granted for a total amount of CFAF 10 076 000 (85% of the loans granted after the first half of 2001). The loans approved were for commerce (82%), agriculture (10%), and livestock production and fishing. In some cases, so-called "emergency credits" were also extended on the basis of informed decision-making. The ceiling for this type of credit was set at CFAF 15 000. Generally speaking, the total amount of credits appears low compared to the amount of deposits, but this is a reflection of the CAPECs' prudent start-up policies and their present level of activities.
The methodology consisted of carrying out a socio-economic study and then negotiating future activities with beneficiaries. Four CAPECs (versus the eight proposed in the project appraisal) have been maintained, owing mainly to the population densities and distribution in the area. The operational approach was divided in three stages, in the course of which extension activities, training and exchange visits resulted in the emergence of the four CAPECs that are functional and recognized by law today. This area remains fragile, however, and most of the general expenses of the CAPECs were still being entirely covered by PRODAM at the end of the project. The total cost for the "rural financing" component was of SDR 943 711, excluding support and infrastructure costs.
Strengthening of capacity. The main results were: (i) training of 214 trainers in 24 different areas, which enhanced their technical, professional, training and communication skills for working in the rural environment; (ii) training in group management and organization for 1 913 rural managers (of whom 37% were women) and formulation, with the beneficiaries, of their own management documents based on their actual needs; (iii) technical training (agro-pastoral and environmental advisory services) for members of EIGs and women's groups in order to enable them to carry out agricultural production, marketing-gardening and women's self-development activities; (iv) literacy and post-literacy training for 9 663 participants (48% women) through 145 centres and production of 42 post-literacy support documents or manuals.
The methodology for providing agricultural advisory services consisted in organizing a participatory assessment in the villages in order to identify constraints and solutions with respect to production activities and then developing the content of programmes. Extension activities included field visits, advice on equipment, demonstration, visits for the exchange of experiences and review days to take stock of progress. As for the activities aimed at the women's groups (more than 4 000 women in all), the intervention methodology consisted in multiplying the training received by trainers and by women leaders chosen by the groups. The women were given instruction in topics relating to reforestation, fabric-dyeing, hygiene, health and nutrition. The total cost for the "strengthening of local capacity" component was SDR 333 431, excluding monitoring and infrastructure costs.
The principal results were the construction of two 113 metre bridges to make it possible to cross the two main branches of the Senegal River at all seasons, two box culverts, five lined drainage channels and two submersible concrete aprons. The amounts budgeted were not sufficient to complete all the works (the lowest bid submitted was CFAF 3 390 008 594, while only CFAF 1 559 000 000 was available). It was therefore agreed that the works that would enable access to the project area during all seasons would be carried out in this first phase, and that the remaining terracing works would be postponed until further financing was received.
The evaluation confirmed that PRODAM has had numerous positive impacts, including, notably, the improvement of productive capital on the farms in the area. The rehabilitation of 1 704 ha of village irrigation schemes benefited 1 484 heads of household and 3 450 women (market gardens) directly, the overall number of beneficiaries totalling 16 324. The project facilitated equal access to land in disparate communities (consisting of owners, refugees, dispossessed persons) and increased fourfold the average amount of land being farmed by the villagers in these areas. While PRODAM did thus enable the most destitute members of the population and the repatriates to have access to the Walo, its approach to land use models for the Senegal River valley requires further study in order to establish its economic sustainability.
The creation of three new pastoral units has opened up 210 000 ha of land for off-season grazing by an additional 95 000 TLUs. While the health conditions and productivity of herds seem to have improved, the breeders' strategy of herd accumulation did not change, and the impact of this sub-component in terms of higher incomes and greater food security was therefore very weak.
Access to water through the boreholes and infrastructure put in place under the project benefited not only the animals but the human population, lightening the daily burden of labour for women in some villages. However, a clearer definition of roles and responsibilities with regard to the institutional management of this infrastructure is needed in order to ensure its sustainability.
Productive capital also increased, thanks to the acquisition of equipment (power pumps) and draught animals via the producer organizations (OPs). Household food security improved as a result of the fact that rice yields on the irrigated lands tripled over five years. Families now have cereal supplies that are twice the food security threshold level set by the FAO. However, these results must be looked at in the light of the findings of a nutritional assessment which revealed that 20% of the children under the age of 5 in the department suffer from acute malnutrition, with no significant difference between PRODAM and non-PRODAM villages. Moreover, it remains to be determined whether this level of intensified farming on irrigated lands is financially sustainability, which is not a certainty, given the evolution of prices in recent years.
With regard to the organization and structuring of populations, the project launched a remarkable experience of promoting the formation of networks and associations, in addition to putting in place institutional tools in support of the project. This fostered the emergence of organizations that have become fora for consensus-building, resource mobilization, beneficiary empowerment, and the creation of future capacity to influence and propose alternatives for sustainable development activities in the area. However, despite these indisputable achievements, some deficiencies persist with regard to the sustainability of the activities undertaken, notably in infrastructure management and maintenance.
Although women play a crucial social and economic role, their involvement in the control of resources and in decision-making is far from commensurate with that role. In terms of social cohesion, the desired impact seems to have been achieved, with 79.5% of the repatriated households having been resettled. From an economic standpoint, net household income has improved greatly thanks to PRODAM and is estimated at an average of CFAF 325 000/ha net per plot. The project's impact on the environment and the natural resource base has also been positive, particularly the decrease in brushfires which has resulted from the assumption of responsibility for management by local organizations.
For all these reasons, the project is deemed to have been relatively effective and responsive to local problems, although, ultimately, the USD 27.6 million directly benefited only 12 villages and 16 500 people, without resolving the problems surrounding equitable and financially sustainable land use in the Senegal River valley or the problems relating to organization of marketing by producers.
The main lessons learned from the PRODAM experience can be summarized as follows:
- Empowerment of grass-roots organizations is one of the necessary conditions for the sustainability and viability of the investments and economic activities supported by the project;
- The empowerment of grass-roots organizations should take place gradually and should be supported by activities to strengthen their capacity for organization and management;
- The intensification of rice production and the diversification of agricultural incomes in the project area will necessarily entail the improvement of access to markets and local financial services that are suited to their needs.
The positive results of PRODAM, the well-developed structure and the effectiveness of the farmers' organizations that have participated in the project, the competence of the project management unit (PMU) and its local partners, and the work undertaken during the participatory evaluation process and the Orossougui workshop are all factors that speak in favour of the formulation of a second phase. This second phase should be highly participatory and should fully recognize the rights and capacities of local stakeholders in rural development.
A grant from IFAD (ECP/NGO) will be made available for NGO "USE" to enable this organization to continue providing support to the CAPECs and the farmers' organizations. In the framework of this grant, the parties will consider extending a budget line of support for the KNB Federation in order to enable it to provide a minimum level of service for its members.
With regard to the communities that are hoping to be included in PRODAM-II, the Federation should remain cautious and avoid taking on commitments that it might not be able to fulfil. As for the farmers' organizations with payment arrears to CNCAS, the Federation should continue to encourage them to settle their debts as soon as possible. In general, the Federation and the CAPECs should continue seeking to work in concert with CNCAS.
1/ The evaluation mission consisted of: Mr Jacques Faye, rural sociologist/geographist (Team Leader) ; Mr Thierno Aliou Ba, sociologist-trainer (facilitator of the participatory evaluation); Dr Haidara Abdoulaye, physician, specialist in community health; and Mr Bouna Camara, demographic statistician. Mr J.Ph. Audinet, Evaluation Officer at IFAD participed in the preparatory mission and in the evaluation workshop. This Evaluation report has been written by Mr Hubert Boirard, agro-economist, IFAD consultant.
2/ 35 000 TLUs for Malandou and Péthiel, 60 000 TLUs for Wendou diohi.
3/ The minimum number of shares is set at one for individual members and four for corporate members.
Agricultural Resources Management Project - Interim evaluation (2004)
Interim Evaluation 1
The Agricultural Resources Management Project (ARMP) was designed in late 1995, and was the fourth IFAD supported intervention in the Hashemite Kingdom of Jordan. The project closed at the end of December 2003, and based on the successful implementation experience the Government of Jordan (GOJ) has requested a second phase of the project. The project was subject to a Mid-Term Evaluation (MTE) in 2000, hence the approach adopted for the Interim Evaluation (IE) was therefore to update the assessment of impacts and the changes that had occurred as a result of the MTE recommendations. In undertaking the evaluation, the field mission fully implemented the Methodological Framework for Project Evaluation (MFE) developed by the IFAD Office of Evaluation (OE) in 2002 and followed its rating system2 .
The overall goal of ARMP, within the GOJ strategies for the development of rainfed farming, was to improve the income stability of vulnerable, resource poor farmers in the two Governorates of Karak and Tafila by safeguarding and improving the productive potential of their natural resources and enhancing their returns to labour. This was to be achieved by arresting soil degradation and restoring soil fertility, promoting activities for women through credit and strengthening the capacity of the Ministry of Agriculture (MOA) both to implement these measures and to meet the technology needs of farmers. To realise these aims t he project interventions were arranged in five components: Resource Management; Agricultural Development; Institutional Strengthening; Women's Development; and Co-ordination and Management. At the time of design the estimated number of rural households living below the poverty line in the project area was estimated at about 7 000. The total project cost was estimated at about USD 18.6 million, of which the IFAD loan provided about USD 12.8 million. The project had a seven year implementation period.
In 2000, after three years of implementation, the project management proposed to revise the targets in the resource management and agricultural development components. These proposals were endorsed by the MTE, and the re-allocations were approved in 2001. The revised targets did not change the nature of the project itself (and hence the objectives), but simply increased and re-balanced the targets on a demand-driven basis. In addition, significant changes were made to the categories for which credit was available: funds were switched from support for supplementary on-farm soil and water conservation (SWC) activities and land consolidation and moved to women's Income Generating Activities (IGAs), where demand was proving to be very high.
Resource management. At the time of the interim evaluation 81% of the revised target for soil and water conservation activities (7 500 ha) had been achieved, covering 2 700 farms. On-farm, the project had constructed 2 871 cisterns of 30 m 3 capacity to conserve runoff and supply supplementary irrigation needs. The cisterns have contributed very significantly to the expansion of the area under tree crops, especially olives, and have greatly improved the productivity of the crop. As well as other erosion control measures, such as gabions and terraces, the project has constructed almost 900 km of stonewalls to protect some 5 880 ha. Stone walls have proved the most popular activity with beneficiary farmers probably because an unintended benefit has been significant increases in land values. Off-farm, 29 rock-faced earth mini-dams and hafiras3 were completed between 1998 and 2003 (58% of the revised target), benefiting some 18 population centres, mainly in respect of livestock watering. The project has undertaken a programme of spring rehabilitation, which involves permanent protection of the source to prevent damage and contamination, and improvements to the canals which distribute the water downstream, benefiting numerous farmers who mostly have very small traditional olive groves. So far works have improved the utilisation of 69 springs (66% of the target), helping some 1 121 farmers.
Agricultural development. New planting has taken place so far on about half the area which has been improved through SWC activities. The overwhelming majority of trees planted are olives (140 000) although about 5 000 pistachio seedlings have also been released from MOA nurseries. The programme of fruit tree rehabilitation has consisted of deep-pruning of old olive trees that had become unproductive, and providing one application of fertilizer. Initially this activity was unpopular with farmers until the benefits were demonstrated, at which point demand greatly increased. Olive rehabilitation normally takes place where springs and canals have been rehabilitated.
Credit4 . Loans have been provided through the Agricultural Credit Corporation (ACC) either for agricultural development or for women's IGAs. At the time of the IE a total of 1 152 loans had been dispersed, of which 1 034 loans have been for women5 (260% of the initial target), hence, over 90% of the total loans, and 80% in terms of absolute value have been provided to women's. Approximately 44% of loans taken by women for IGAs financed dairy processing (with sheep milk), whilst other activities have mostly centred on other forms of food processing. Overall, the loans appear to have been very profitable (more so than at the time of the MTE): the mission calculated that average annual profits were JOD 829 (USD 1 160) or 57% of average annual household incomes. The result of stringent lending conditions has been a high repayment rate (89%). By June 2002, 31% of resources allocated for the institutional strengthening for ACC had been utilized, but ACC remains reluctant to use these loan funds, arguing that they should be provided as a grant. This has meant that ACC's outreach in the project area has not changed.
Women's Development Programme (WDP). Despite insufficient staff numbers or external support, the WDP has evolved into a much bigger programme than expected. Three NGOs and two universities have been contracted to provide training for the beneficiaries, and by project completion, 343 training workshops had been conducted (89% of target), reaching approximately 2 526 women. This is most impressive. However, the mission concluded, in agreement with the MTE findings, that the majority of WDP beneficiaries were not drawn from the poorest groups.
The WDP officers have found resistance to the concept of small enterprise groups, and none have been formed through ARMP. The mission concluded that the WDP staff neither had the appropriate skills nor did they have suitable training and support to introduce this concept.
Institutional strengthening. The project has arranged a very comprehensive training programme for staff, which has been very useful and will have long term benefits. However, there has been reluctance to use loan funds for more innovative aspects of the project, including hiring NGOs to provide support in specific skills, and to undertake studies. The mission considered that this had curtailed the extent of potential benefits and the development of new capacities and more participatory approaches.
Coordination and management. The IE considered that implementation arrangements for the project had generally been very satisfactory, with adequate coordination and management procedures. However, participatory approaches, aspects of community development and involvement and group formation have not been adopted, even after very strong recommendations made by the MTE and agreements reached in the ACP. This is unfortunate, as GOJ policies and MOA strategies now require exactly the sort of approaches described in the original design of ARMP.
Relevance. Overall, the mission considered that ARMP's SWC and agricultural objectives remained very relevant to the intended target groups. This conclusion was derived partly from considerations of the impacts on poverty presently being achieved, a summary appreciation of the poverty situation prevailing in Karak and Tafila, and GOJ's agricultural and rural development policies. If the credit activities had been implemented as designed, then their continuing relevance to the intended target groups would also be high. The success of the project has in many ways just highlighted the considerable potential which still exists in the two governorates. The relevance of objectives is rated as high (4).
Effectiveness. Overall, the mission concluded that the objectives of the project had been significantly achieved. Major improvements have been recorded in soil and water conservation, agriculture, and skills levels. The project completion report shows over 97% of physical targets achieved. By upgrading and protecting the natural resource base, through technical support to farmers and by providing credit and skill development programmes (especially for the women) the vulnerability of a section of the rural communities has been decreased. The re-allocation of targets and resources after the MTE improved the potential effectiveness of the project by better aligning targets and funds with beneficiary demands and project capacity. The mission concluded that the performance of the project in terms of its implementation effectiveness was high. The only detracting factor has been the lack of relative poverty of the groups that had been reached, which, it was further concluded, partly reflected the practical difficulties in targeting the interventions. The evaluation rated overall project effectiveness in achieving its objectives as substantial (3).
Efficiency. At appraisal, the incremental benefits were projected to be derived only from crop production, and the Economic Rate of Return (ERR) was estimated at 10.6%. The mission considered that this potentially underestimated the project benefits, as benefits are also derived from off-farm water storage structures, and the credit programme (through the promotion of IGAs). When all productive activities are combined, the Project's revised ERR was calculated by the mission to be 34.8%. This is a very good result, (which is not unduly sensitive to price/cost changes) and indicates that borrowing for ARMP development activities is well worthwhile and should lead to increasing economic returns for Jordan . This would also suggest that the project has been efficiently implemented. Improvements to physical resources, enhanced and rejuvenated agricultural production, and promotion of off-farm income generation have also proved positive for family incomes: the likely increases to farm household incomes indicated in the Appraisal (between 30% to 400%) are probably correct. The evaluation rated the efficiency of the project as high (4).
Physical and financial assets. Households either adopting SWC activities or benefiting from spring rehabilitation have seen their physical assets improved, probably best illustrated by the increased values now placed on their holdings. Impacts on household and financial assets have been reported by beneficiaries as a result of the IGA programme, and mission calculations indicate that these impacts are significant. Approximately 1 800 households have been given improved access to financial services through the credit programme. These improvements are impressive and are considered sustainable. Access to markets - an important factor in poverty reduction - has improved since the beginning of the project, as a result of beneficiary (and other) demand. However, the mission concluded that beneficiaries of the SWC, farm development and credit programmes were mostly in the middle-income bracket (in the context of rural incomes), therefore, although the project has had an impact on assets, it had less impact on poverty than it might have had, if it had been able to target the poorest sector of the rural population. Nevertheless, the overall rating for this sub- criterion is high (4).
Human assets. Access to potable water has improved either directly as irrigation facilities have been developed, or indirectly as a result of increased incomes. The project activities, by their nature, have resulted in increased workloads, both for women and children, as activity levels have been raised. All beneficiaries have either received skills training, or have profited from increased access to information via the project staff. Overall, the rating for this sub-criterion is substantial (3).
Social capital and empowerment. Although the design called for participatory approaches, in practice these have not been adopted. Because of the lack of participation, social cohesion and local self- help has not been an outcome of ARMP, neither has the project made any contributions to building local capacities for community empowerment, nor in fostering other self-help initiatives. Overall the rating for this sub-criterion is negligible (1).
Food security. The project has changed farming technology and practices by increasing farmers' use of fertilizer for tree crops, increasing the amount of irrigation water, more effective pruning, increased use of contour ploughing, and more rational cropping patterns for land of varying gradients. As a result land areas and productivity have increased, and the combined effect will be to improve the robustness and resilience of the farming systems, thereby improving food security. The overall rating for this sub-criteria is therefore high (4).
Environment and communal resource base. Changes in the natural resource base have largely been positive as water storage facilities constructed by the project have resulted in improved pasture, and the area of protected and stabilised soil on private farms has increased as a result of the SWC measures and the expansion of tree cropping. However, the project-sponsored cisterns for supplementary irrigation requirements do not adequately serve the new areas planted to olives and unintentionally the result is the depletion of what is already a severely depleted aquifer. On balance, however, the project has reduced short-term environmental risk and thereby contributed positively to livelihoods. The rating is substantial (3).
Institutions, policies and the regulatory framework. No significant changes in public institutions policies or service provision have resulted from the project's activities. The major change required is modernisation of the inadequate extension services, and ARMP has served to highlight this need. As a result of the credit experience ACC has been willing to modify its lending terms and conditions somewhat, however the changes made are small, and still do not mean that the poor will be eligible for credit. No substantial contribution has been made in promoting gender sensitive policies. Overall, the evaluation concluded that the impact of the project on institutions and policies had been limited, and the rating is therefore modest (2).
Gender equality. The project design included not only the women's programme, but specific references for inclusion of women-headed households in the SWC and agricultural components to enhance women's gender equality. As a result of demand, the credit allocations for women's IGAs have been significantly increased, which has also had the effect of increasing the skills training offered to women. Overall, therefore, women's access to project benefits has been good and more than half of project beneficiaries are women. Women beneficiaries confirmed that they felt more empowered and played a stronger role in decision making in the household as a result of being income earners, particularly those that had no previous employment. However, the project has not contributed to or facilitated progress in gender sensitive policies (paragraph 18). On balance, the impact on gender equality is rated as high (4).
Sustainability. The IE concluded that the project SWC interventions would be largely sustainable after project closure, as the on-farm structures require little maintenance. By comparison, there are significant maintenance requirements for the mini-dams, hafiras, and check dams on State land, and it is probable that the long term maintenance of these structures will remain a government responsibility, with little possibility of cost recovery from users. As benefits increase from the planting or rehabilitation of tree crops, then annual inputs from the farmers are likely to continue and will maintain yield levels. The sustainability of the increased area under olives may be threatened by low prices for olive oil or high prices for groundwater.
Although ACC has not established a specific revolving fund it has confirmed its commitment to continue to provide loans by going beyond the initial allocations for credit. The credit line is profitable for ACC and there is a demand for it, hence there is no reason why it should not be continued as part of ACC normal/ongoing operations. Mission estimates are that despite some incongruence between loan duration and actual funding requirements, the IGAs undertaken are in general profitable for the borrowers, and hence will be continued. On balance, therefore, the evaluation concluded that the major project activities would be largely sustainable, and that the overall rating should be substantial (3).
Innovation and scaling up. The project involved only limited innovation, rather success has been achieved by making known improved practices available to more people. This is replicable – hence the request for a second phase. The deep pruning of old olives can be described as a new technique, which can be replicated through the extension services wherever it is appropriate. To an extent, the introduction of check dams was also innovative. The project has provided training to many women who could not take loans because they could not meet ACC's conditions, if loan conditions could be modified then this programme could easily be expanded. Overall, the rating for innovations and scaling up is modest (2).
Other poverty impact. The emphasis for development within GOJ has shifted more towards ways of achieving poverty alleviation and creating employment – a strategy to which all Ministries are expected to contribute. For MOA, the successful experience with ARMP is the most appropriate initiative on which to build to help realize this strategy. Whilst this affect may not yet have been seen, it promises to be a significant factor in future planning. The rating for this sub-criterion is modest (2).
Overall, rural poverty impact as a result of the project was rated as substantial (3).
IFAD and the Cooperating Institution (CI). In assessing IFAD's performance, the mission concluded that whilst the intentions and relevance of the interventions could be rated very highly, there were some aspects of the design which were poor – especially the targeting criteria and the practical aspects of the implementation approach for community involvement. The arrangements for the involvement of ACC also did not offer any progress in terms of enhancing the outreach of credit to the poor. Overall, IFAD facilitated a design which "did the right things", but did not systematically ensure that "the right things were done right".
Overall, the mission concluded that the CI had performed well in supporting project implementation, and decided to rate the performance of both IFAD and the CI as substantial (3).
Government and its agencies. Overall, the arrangements made by MOA for project implementation have proved very satisfactory and commitment to the project is high. The only difficulties experienced during implementation relating to the MOA have been the reluctance to change Ministry operational arrangements to meet the implementation mechanisms as specified in the loan agreement and the reluctance to use loan funds for Technical Assistance (TA) or for contracting skills through NGOs etc. The Project Management Unit (PMU) has proved a very effective mechanism for project implementation, which has been demonstrated not only by the physical achievements of the project, but by the very pragmatic way that problems have been overcome, the decisive management involved in the modification of project targets at mid-term and the very good M&E arrangements and output. The ACC has been an effective partner in terms of disseminating the credit funds, but cannot be said to have contributed much to the achievement of the project's poverty objectives. Overall, the performance of the government agencies is rated as substantial (3).
Participation and targeting. In practice there has been little beneficiary participation in project planning or implementation. In its recommendations the MTE provided detailed suggestions as to how progress might be made with addressing participation in the second half of the project, and after this a community-based approach was developed in the spring and canal rehabilitation activities. However, no such approach has even been tried in the WDP, where it could have encouraged the participation of poorer women. The IE came to the conclusion that the targeting mechanisms in ARMP had been weak. Inclusion or exclusion as a project beneficiary had become both confusing and rather arbitrary, and there did not seem to be many factors which tilted the benefits towards the poor.
SWC and agriculture. The evaluation concluded that there had been substantial achievements in most of the major objectives of both the SWC and the agricultural development components. Firstly, the degradation of land and water resources has been reduced or arrested on both public and private land where project interventions have taken place. Secondly, there has been a shift from cereal to tree crop production on sloping land, with the consequence of improved conservation and the potential for improved farmer incomes, and thirdly, rangeland grazing and water access has been improved. However, the objective of optimising the long-term productive use of the land and water resources still needs to be addressed in the light of the continuing dependency of most project farmers on abstracted groundwater. In addition, the sustainability of the improvements achieved so far may be affected by reduced prices of olive oil, the reliance on purchased groundwater, and the lack of formally structured Water User Associations on communal irrigation schemes.
Credit. In terms of the disbursement of credit funds, ARMP had been very successful and had had a significant impact on household incomes for over 1 000 beneficiaries. The high repayment rate on these loans has meant that the exercise has been profitable for ACC and is a good indicator of sustainability of the programme. However, the mission concluded that one reason for the high repayments had been the stringent lending conditions, which meant that borrowers predominantly came from non-poor groups. The intended institutional strengthening and other support for ACC have largely not been realised, with the result that ACC's has not extended its outreach as planned.
Women's programme. The evaluation concluded that the WDP had achieved its objective of increasing family incomes through IGAs. The outreach has been very creditable, especially given the under-staffing of the Unit, and has demonstrated the popularity of and need for the credit which was made available. In addition, the provision of skills training pre-credit has enhanced the capacities of and served to empower a very large number of women, much more than was ever planned for in the design, and this will bring its own longer term benefits, even if credit is not made available to all.
Institutional strengthening. The very comprehensive training programme provided for project staff had been instrumental both in increasing capabilities and fostering good team spirit. Similarly, considerable training has been provided for beneficiaries. However, whilst skill levels may have been deepened, they have hardly been broadened, as little use has been made of the funds included for NGO support, research and studies. This has acted to limit the implementation of the project in the ways envisaged in the appraisal, especially in terms of developing participatory approaches and widening the scope of IGAs.
Coordination and management. The evaluation concluded that the project implementation mechanisms had largely been very effective, and had contributed both to the extent of the project achievements and the dynamic nature of the processes created. However, there is a need to arrange for a handover mechanism, so that completed areas and activities are on-passed to District based officers for routine follow up operations. In terms of coordination, the evaluation foresees a need to strengthen the role of the ACC as an implementing partner. At present credit activities are divorced from the central soil and water conservation objectives and activities and the PMU and ACC manage their separate aspects of the project on parallel tracks, without clear common objectives.
Participation. In the second phase ARMP could make a significant contribution by assisting MOA to apply participatory approaches in its poverty alleviation and agricultural development efforts. What is needed is the adoption of a strategic approach to participation and its mainstreaming at all levels within the Ministry. The mission has prepared a series of steps to guide this process (see recommendations, Chapter VIII). In order for the poverty focus of the project to be improved it is necessary to sharpen and clarify the targeting mechanisms. In the design of the new phase consideration should therefore be given to directing resources specifically towards the poorer groups, learning from the experience of phase I. A lost opportunity in Phase I was the lack of reaching the target group with credit for IGAs, and phase II needs to consider alternative strategies.
Water. In phase I the efficiency of water use in agriculture was not given the highest priority, but t he limiting factor in GOJ's development strategy for agriculture through conservation is water, not land. Firstly, this means that the project must stop relying on ground water for irrigation and gear the extent of the agricultural area to the scarcest resource, i.e. water. Part of the reason for the present groundwater abstraction is that the present capacity of the cisterns (30 m 3 ) is almost invariably too small for the supplementary irrigation requirements. However, the evaluation found that the present project proposals to address this problem may actually serve to exacerbate it, and are not based on adequate examination of the irrigation needs. Secondly, there was considerable variation between farmers in the frequency of irrigation and in the amounts of water applied, suggesting that both over-irrigation and under-irrigation may occur. In a low-rainfall area farmers, irrespective of their level of experience (and it should be borne in mind that some of the project's beneficiaries come from non-agricultural backgrounds), need to be given clear guidelines and recommendations regarding efficient water-use practices. Thirdly, in order to protect the investments already made and ensure a mechanism for management and maintenance on community schemes, and to settle any disputes which may arise, it is vital to ensure that there is an established local organisation which will take responsibility for the completed works. The present systems were found by the mission to be largely inadequate, and in phase II this needs to be given high priority. To address these three issues in water, the evaluation made a number of recommendations for consideration in phase II (Chapter VIII).
Agricultural extension. Jordan's agricultural extension service is seriously under-staffed and its effectiveness is consequently compromised. This presents a problem in terms of the continuing support required by farmers in order to fully realise the development potential from improved land and water resources. At the moment the extension agents working with the project appear to have to continue their activities with an ever-growing portfolio of clients. This is addressed under the decentralisation proposals of the evaluation (Chapter VIII).
Credit for development. In considering a second phase of ARMP, credit needs to be given a more central role and to be considered in the wider view of rural financial services, following IFAD's policy guidelines (which had not been developed a the time of the original design). ACC should continue to play a key role in any MOA development strategy, as it remains the only formal organisation able to provide financial services to producers in the rural areas. The need, however, is to identify the mechanisms through which ACC can grow into a much more broadly based lender, able to reach out to those individuals and groups who need ACC services in order to climb out of their poverty. Despite the achievements and successes in phase I, the limitations demonstrated show that there is still some way to go.
The evaluation has identified three issues on which progress needs to be made. These are, firstly, arranging for better integration of development objectives and coordination of activities between the PMU and ACC branches, as well as providing a mechanism for monitoring credit utilization. To achieve this it is recommended that a Credit Unit is included in the PMU. Secondly, the possibility of lending through groups must become a reality, at least for a small proportion of the credit funds provided. And thirdly, further institutional strengthening should be provided for ACC, to share with the Corporation the burden and risk in accepting the challenge inherent in making changes to their credit mechanisms. ACC needs external expertise and stimuli to establish the new mechanisms suggested, and needs support from an organisation that has positive experience in anticipating and addressing the difficulties that will arise. Detailed recommendations are made by the evaluation covering these aspects (Chapter VIII).
Women and poverty. Whilst the present WDP provides a building block for phase II, it should not just be copied. Rather, the need is to improve a number of aspects. In particular, in order to reach poorer women it will be necessary to encourage group formation, before moving to suggesting IGAs and providing credit. To facilitate group formation will require the assistance of a skilled and experienced local NGO, which will need to emphasise capacity building for beneficiaries (such as the organization of collective working, administration and marketing skills). The evaluation also suggests the sponsoring of women's centres, as it would be more cost efficient for poor women to work collectively and would assist in institutionalising women's groups. Finally, the range of IGAs needs to be expanded after feasibility and marketing studies which seek to identify products that have market demand and are financially viable.
Institutional strengthening. In phase I the Ministry was not convinced that the participatory approaches being suggested were either cost effective or appropriate to the development being sought. However, it is now timely to help the MOA with the sort of institutional strengthening which will let them develop new approaches to development. The key to this is for the Ministry to accept the need for assistance with the development of skills which it does not have. An appropriate package of institutional strengthening should therefore be included in phase II, covering both the technical training requirements of beneficiaries and Ministry staff, and sufficient to allow for the adoption and introduction of the suggested participatory methods. In addition, assurances should be sought that identified staff needs and planned training activities will be fully funded during implementation of phase II.
The evaluation considered that in phase II wider use should be made of the assistance and support available in other Departments in the MOA, both to address strategic issues (e.g. the marketing of olive oil) and to obtain technical support (e.g. with arranging training). In addition, care should be taken to link with the Ministry's own M&E Department, in order to harmonise reporting on the overall progress of projects. Consideration should also be given to the adoption of the M&E system devised for ARMP in a wider context: there is the potential to extend the methods used to many projects in the Region. With appropriate support ARMP could provide a local "centre of excellence" for training in M&E.
Decentralisation. The differentiation between project activities and the regular roles of the staff of the FDAs have been blurred. Whilst this may have come about because the project has more funding and is able to promise more to potential beneficiaries, it is not sustainable. Essentially the strategy should be for the project activities to temporarily complement the activities of the FDAs. This may require additional support for the extension services as well as a formalised handover process by which developed areas become the responsibility of the FDAs. In addition, the ongoing involvement of the Regional Agriculture Coordinating Committee (RACC) needs to be strengthened, so that relevant decisions about the project's day-to-day work programme can be taken by the agencies most closely involved. Both these aspects would serve to reinforce the decentralised nature of the project's activities. As such, the future purpose of the PMU might then be better understood as demonstrating the new approaches and techniques and training the local staff, before moving on.
Recommendations to this affect have been provided.
1/ The Interim Evaluation Mission was composed of Mr Frank Butcher , Mission Leader and Institutions Specialist; Dr Mouna Hashem, Sociologist and Gender Specialist; Mr Swithun Goodbody , Natural Resources Management Specialist; Mr Omar Imady, Credit Specialist; Mr Sarath Mananwatte , Economist and Financial Analyst. Dr Mona Bishay , Deputy Director of the Office of Evaluation undertook the preparatory field mission, supervised the evaluation and led the process of presentation of the Aide Memoire in the Wrap-up Meeting.
2/ Methodological Framework for Project Evaluation (EC 2003/34/W.P.3)
3/ Hafiras are reservoirs used for livestock.
4/ Credit was not a separate component of ARMP, but was the major means of achieving project objectives. Credit activities were implemented by the Agricultural Credit Corporation (ACC), which is part of the MOA, through a subsidiary loan agreement.
5/ This was the figure for end-June 2003.
6/ The evaluation methodology followed (see Appendix 2) uses three composite evaluation criteria: project performance (composed of relevance, effectiveness and efficiency) rural poverty impact (composed of six impact domains, sustainability, innovations and gender equality) and performance of partners (including IFAD, implementing agencies, and the cooperating institution). The ratings used to assess performance using these criteria are high (4), substantial (3), modest (2) and negligible (1). For the sustainability criterion the rating used is highly likely (4), likely (3), unlikely (2) and highly unlikely (1).
Rural Micro-Enterprises Project (2004)
The rural micro-enterprises project (PROMER) began activities in June 1997. Its total cost is 10.94 million de USD, with 67% covered by IFAD loans (n° 402-SN and SRS-47-SN). Its initial duration of six years was extended by two years in 2002, without additional budgetary allocation. The closing date for the IFAD loans is 31 March 2005. A possible second phase is being considered by the government and IFAD following the evaluation of the IFAD programme in Senegal done in 2003 and the focal points of the intervention as set forth in the Country Strategic Opportunities Paper (COSOP). Given this outlook, the IFAD Office of Evaluation (OE), in accordance with standard procedures, conducted an interim evaluation of PROMER.
According to a joint partnership evaluation, the objectives of the interim evaluation mission were: i) to assess the suitability, effectiveness and sustainability of the actions undertaken by PROMER, ii) to understand, estimate and, if possible, measure the impacts of the project achievements and changes made, and iii) to assess the current dynamic of changes made and their projected sustainability, and iv) draw useful information and recommendations for the future.
The evaluation mission1 stayed in Senegal from 25 January to 18 February 2004. It facilitated a self-evaluation by the PROMER team, and conducted field visits2, held meetings with partners that may or may not have been involved in project implementation and held a workshop on restitution and exchanges in Kaolack based on preliminary results.
PROMER exists within a context of State withdrawal and of encouraging the formal or informal private sector to take over the productive and commercial activities. Micro-enterprise thus proves to be a resource that can make it possible to reduce poverty in a rural environment. The off-farm processing and service activities represent an alternative to the rural exodus for young people and offer women the possibility of carrying out paid activities, especially productive activities that could enhance their knowledge.
PROMER targets development of off-farm micro-enterprises in a rural environment in four regions of Senegal: Tambacounda and Kolda, at first, expanding to Kaolack and Fatick at mid-term. These four regions, affected by the groundnut crisis and displacement of the cotton-producing zone, represent approximately 51% of the national territory and a population of some two million. Although they do offer agropastoral resources, there is isolation and chronic underdevelopment, especially in Tambacounda and Kolda.
The PROMER objectives were defined as:
- To create new paid seasonal or permanent jobs, thus improving income of poor rural families;
- To increase production of quality goods and services by enabling local farm production;
- To extend the annual productive work period beyond the farm work period;
- To reduce the rural exodus using work opportunities for young people.
The target groups of the project were to be formed of individuals or of economic initiative groups. Women and young people were to be priority groups and the poorest segments of society were targeted in particular by support in forming micro-enterprises to represent 70% of the rural micro-enterprises supported. In quantitative terms, the target group was estimated to have 4 000 people, 30% of whom received project support, or 1 200 micro-enterprises held 50% by women. Approximately 1 000 of them should receive funding for a total of some 800 million CFAF and 200 just training. There were an estimated 3 000 jobs expected.
Implementation was to use a progressive approach and develop a mode of intervention that used subcontracting (outsourcing). The intervention was planned for two regions at first, Tambacounda and Kolda, with an expansion at mid-term.
To achieve its objectives, PROMER implementation relies on the joint effects of two main components: (i) support for development of rural micro-enterprises, and (ii) funding for rural micro-enterprises where implementation is done by a microfinance institution, CMS, project co-sponsor3. A third component provides project monitoring and management.
Placed under the oversight of the Ministry of Agriculture and Hydraulics, PROMER is supervised by a steering committee. The relatively light project management unit (PMU) is accompanied in each of the four intervention regions by a regional cell4.
As of the mid-term review, conducted by the cooperating institution, the West African Development Bank (BOAD), the project did a major reframing of interventions reflected in abandonment of support for rural micro-enterprises with predominantly trade and pasture activities, introduction of a structuring cluster approach, diversification of partnerships in the area of financing, and reframing the missions of the multipurpose economic outreach agents (AEP) into business consultants.
After approximately six years of activities, on 31 December 2003, a total of 3.78 trillion CFAF had been disbursed out of a projected total of 5.252 trillion CFAF, or a disbursement rate of 71.97%. It should be emphasized that he government regularly provided its counterpart funds and also doubled its contribution to project funding, especially for the training component.
PROMER has a total portfolio of 1 359 rural micro-enterprises including 683 new creations, or 50.26%, and 676 consolidation rural micro-enterprises distributed among 675 villages with significant dispersion5. 33% are groups with 74% women. Women are 45% of the sponsors of all rural micro-enterprises and young people are 26%.
There is a rather broad diversity of activities, with some dominant ones, which are bakeries, 12% of rural micro-enterprises, soap production, 11% of rural micro-enterprises, miscellaneous trade (11%). The mechanical metal products group represents only 10% of the portfolio.
Also, 1 039 rural micro-enterprises, or 76%, are now considered as operational. 65% of these operational rural micro-enterprises, or 672, are currently monitored by the outreach agents.
The jobs created and consolidated total 3 630, with 1 888 creations and 1 742 consolidations, or an average of 1.4 jobs in the individual rural micro-enterprises created and 1.6 jobs in the rural micro-enterprises consolidated. The rural micro-enterprises with group status total 2 514 jobs, or 69% of all jobs considered and 83% of jobs created. In these groups, there is a distinction between jobs corresponding to the participation of members in the collective activities6 that the mission called "job equivalents" and those corresponding to paid salaried jobs. According to these characteristics, the mission estimates that jobs created and consolidated to date would be approximately 1 5007.
The sectors with the most jobs (except for job equivalents) seem to be according to the number of jobs per unit: processing of cereals and transport (2.1), restoration (2.3), valves and woodwork (2.6), metal construction (2.7), fishing (3.1), tourism (3.3).
Training is the main tool for reinforcing capacities and skills used by PROMER. Thus, 943 rural micro-enterprises received at least some training, or 69% of all rural micro-enterprises for 2 213 participants, not including literacy. 64% of participants received technical training, some of which was provided by artisan trainers, and 36% received cross training (entrepreneurship, business management, etc.). Women are 55% of those receiving training. The functional literacy training was given to a total of 2 226 people, 82% of them women. Less than one person out of two trained (47%) implemented the training. Access to funding to buy equipment seems to have been a significant constraint in applying the technical training.
The other non-financial support mainly involved
advisory support provided by the outreach agent. Over the past three years, they conducted 13 076 visits to the rural enterprise economic interest groups, or a monthly average of 16 visits per agent8 for an average active portfolio of 28 rural micro-enterprises. The intensity of monitoring is not the same for all rural micro-enterprises, but about fifteen per agent receive more regular monitoring, about two visits per month. These are rural micro-enterprises that are easily accessible and close to the agent's work site.
commercial support addressed, among other things, market research, visibility of the rural micro-enterprises and product packaging. The exact, cumulative quantification of this support is not known, nor are its results. The project also supported participation of some rural micro-enterprises in trade fairs and, in February 2003, organized its introduction of the Initiatives Mobile Unit, which recorded the permanent participation of 12 rural micro-enterprises and 5 000 visitors. In 2003, the actions focused on creation of two rural micro-enterprise boutiques. These boutiques are promoted from the perspective of project withdrawal and the necessary sustainability of the rural micro-enterprises.
the technical support targeted improving the traditional production processes and support for installation of production units, especially for the rural micro-enterprises involving processing of agricultural and forestry products. Seven clusters were involved with adapting equipment, several clusters also received improvement in routing. A centre for information on documentation and demonstration (CIDD) was created in 2002. The project also introduced two tools in 2003 to allow the project to take over some of the risk for the technological innovations (for equipment or packaging): the technical support fund (FAT) and the commercial support agreement (CAC). Socioprofessional associations (seven have been identified) were encouraged or promoted under the project as of 2000, although the level of autonomy of these organizations is still far from satisfactory.
Regarding financial support, the project contributed to reinforcing capacities of partner institutions and also coaches the micro-entrepreneurs to obtain a loan or to set up self-financing.
Although the onlending agreement for the line of credit and the memorandum of agreement were signed with CMS, the line of credit never received a call for funds from that institution. However, the risk funds that were to cover some of the losses to loans9 were mobilized in two parts and CMS received some equipment. CMS did not open the 10 new funds projected but rather during the course of the project closed some funds or transformed some into points of service. This situation led PROMER to seek new partners. Two new subsidiary agreements and memoranda of agreement were thus signed in 2003 with l'Alliance pour le crédit et l'épargne pour la production (ACEP – production savings and loan alliance) and l' Union des mutuelles d'épargne et de crédit de Sédhiou (UMEC – Sédhiou union of mutual savings and loans) which received a contribution of equipment. Distribution of the line of credit should be reassigned, in the amount of the mobilization, and is now as follows: CMS (60% of 520 000 SDRs, or 312 000 SDRs), ACEP (30%, or 156 000 SDRs) and UMEC (10%, or 52 000 SDRs). At the end of 2003, the two latter institutions made a first call for funds and mobilized the risk funds. The project also signed memoranda of agreement with four local microfinance institutions and made some equipment (motorcycles or computer equipment) available to them.
Based on these various partnerships, 770 rural micro-enterprises obtained at least one loan, or 57% of rural micro-enterprises in the total portfolio, for a total of 265.3 million CFAF corresponding to 983 loans. Also, 1 422 accounts were opened at the partner decentralized financial services and 1 011 savings plans were formed. Women (individual rural micro-enterprises and groups) received 37% of the total allocated and 46% of the number of loans granted. The "processing of agricultural and forestry products" and "service and other" clusters received most of the funding, 35.1% and 42.6%, respectively.
Despite the absence of mobilization of the line of credit, CMS granted 84% of loans in volume and 88% in number, ACEP granted 9% in volume of loans and 4% of the total number. Approximately 129 rural micro-enterprises, some of which received a loan, also participated in self-financing of their equipment (space planning, miscellaneous equipment with extremely variable costs, etc.)
It should also be noted that PROMER furthered emergence of a new trade through 24 advisors who could privately and/or under a second phase of PROMER participate actively in coaching the rural micro-enterprises and the project management.
The quantitative objectives projected regarding the number of rural micro-enterprises in the portfolio were achieved. Those related to training were partially achieved. On the other hand, considering the financial tools available, the objectives regarding job creation, especially for young people, do not appear to have been achieved, and the same is true for access to funding for the rural micro-enterprises. Introduction of new equipment and innovative processes is positive and is one of the major PROMER accomplishments.
Generally speaking, the project team was enthusiastic and engaged in its mission. The project group worked satisfactorily overall and there were significant results in defining a strategy for intervention and for testing various tools. However, some of these tools pertain more to promotion than to sustainability (mobile unit, CIDD). Others such as CAC, FAT and the reference rural micro-enterprises are in the current stage of tools for research/action, technical suitability and demonstration in some promising clusters. It seems relevant that these are medium-term routes to sustainability but this approach currently reaches only a limited number of rural micro-enterprises, which will restrict interpretation.
The project is sometimes a replacement for the rural micro-enterprises in some functions by insufficiently associating and empowering the rural micro-enterprises or their organizations: procurement and sales market research and contracting, and cash flow management.
Regarding financial support for funding equipment over the medium term, the project faced the absence of suitable financial products at the level of CMS. This lack made it impossible to achieve some of its targeted objectives. The initial financial tools strategy seemed inadequate: the conditions for entry and access to credit, including the distance of the services from the main partner institution, immediately eliminate part of the target public, particularly jobless young people and the disadvantaged women. These tools sought to reassure the partner institutions and above all to make resources available to them to grant loans but they do not affect the capacities of the target public to meet the conditions imposed by these institutions. Additionally, if management of the financing was effectively externalized to the decentralized financial systems, most of the other functions (information, assembly of files, even monitoring of retrieval) were taken over by the outreach agents to the detriment of the essentially advisory functions.
The approach of measuring efficiency, as a pilot project, should be taken cautiously, and some ratios show very unequal results but for rural micro-enterprises the return on investment is quite rapid; they can generate a supplementary operating result that is two times as high as the average coaching costs (direct and indirect).
The mission showed diversified impacts localized at various levels of observation (rural micro-enterprises, household, village rural community) and involving different domains (economic, social, etc.).
Overall, positive impacts, even quite impressive ones, are shown for some micro-enterprises that received consistent support. The effects are seen at the level of goods and services as well as in sales. There are cases in which the volume of activities doubled (sometimes tripled) within one year of activities. The impacts are strongly tied to the intensity and duration of coaching for these rural micro-enterprises. Nonetheless, the impacts remain modest at the scale of the local economy. Supplementary income disbursed, especially by female entrepreneurs, is allocated on a priority basis to improving the food situation for the family, with approximately 1 350 households having been affected by this impact. When the surplus is significant, it is invested into growing the productive capital of the family farm, especially for agricultural production and diversification of off-farm activities. This thus allows for better security for the households involved, better access to primary health care, and, to a lesser extent, higher school enrollment for children and improved living conditions. Potentially, 860 households showed changes at this level. There is a reduction in the difficulty of work for women because of the processing equipment introduced, and it can be estimated that about 2 000 people in all categories benefited from this impact in the project zone. The skills acquired at the professional level also made it possible to have an impact on human resources. The financial success of some sponsors led them to play a more significant social role in the village and, in some rare cases for those with the greatest success, they are involved in local development of the rural community by putting their reinforced capacities and their resources to use in the community (funding school meals, furnishing schools, etc.). The impacts in terms of environment can be classified as minor and indirect in most types of rural micro-enterprises.
The appropriation and dissemination of certain technological innovations, such as an improved oven for bakeries, seem to have been achieved. On the other hand, other innovations are still under experimentation (shea butter press, improved press for groundnut extraction, etc.). For the latter, it is not currently possible to say whether they will eventually be adopted.
From now until the end of the project, it cannot be said that all of the currently operational rural micro-enterprises are on the path to sustainability, due mainly to insufficient coaching in terms of duration and coaching that is insufficiently empowering and participatory. Thus the impacts and overall effects are limited. Although the impacts shown are quite significant for some of the rural micro-enterprises but overall relatively limited, within the framework of a pilot phase, they make it possible to assess the impact of the methodology and the interest of an approach that associates training with advisory support and access to credit.
The reorganization done with the mid-term review made it possible to take into account sectors of activity that had been insufficiently supported until then. However, the systematic abandonment of trade and pasture activities was not appropriate given the objective of reaching a disadvantaged public for whom these activities generate income and could be a spark to activate more structuring activities. The trade activities could also accompany development if production activities by furthering marketing. Thus, the types of activity targeted, the conditions for access to credit and the insufficient proximity of financial services, in some zones, are discriminant factors for the target priority public, women and young people not involved in off-farm activities.
The private and associative partnership was active, dense and diversified, which made it possible to execute an ambitious programme of work within a relatively short time that was wide geographically and sectorally, even if there is a regrettable content of overly uniform memoranda of cooperation that did not always make it possible to make best use of each partner.
The choice of an intervention over all four regions was of some interest due to the pilot nature of the operation and the diversity of the potentialities and types of activities but cold have been limited in each region to concentrations of economic activity without this affecting the choice of the priority target of the project. This would have made it possible to avoid the scatter effect noted by the mission and thus to reduce operating costs.
The experimentation under way since 2002 on the so-called structuring clusters has interesting lessons that can be capitalized for the future. However, the proximal procurement and sales markets were exploited very little although they involved nearly all of the rural micro-enterprises from the portfolio and had constraints to be lifted. In its apprehension of the rural micro-enterprises, PROMER seems to be adopting a sectoral rather than a cluster approach. Support is focused on rural micro-enterprises that, alone or nearly so, carry out almost all of the functions (procurement, production and sales). They more or less involve the micro-enterprises upstream and/or downstream of the rural micro-enterprises supported. This strategy, which seems to fall within a vision of evolution of the affected rural micro-enterprises towards formal small and consequential enterprises seeking to cover the high value added markets and modern distribution, contributes to limiting the impact of the project and the driving role of some rural micro-enterprises.
Although the sustainability tools chosen by the project (CAC, FAT, rural micro-enterprise boutique) seem relevant, their experience in the allotted time (from now until the end of this phase of PROMER) and at the rate of introduction noted cannot, however, allow for assessment of their suitability to the targeted sustainability objective. It would thus be desirable in the future to be able to test and build on the bases for sustainability of these tools in a real situation with a completely participatory, empowering approach.
The mission shows that the strategy for project withdrawal and for sustainability of the advisory support functions for the rural micro-enterprises was not in-depth enough at the time of project design and was not considered thoroughly enough during the course of the project, which would have made it possible to test one or more of the options within the framework of an experimental operation.
The mistakes and insufficiencies in design of the monitoring and evaluation system throughout the entire period of the project, added to the instability of staffing for this service, made it impossible to optimize the guidance of project interventions. These insufficiencies do not currently allow for optimal capitalization on experiences and for a good measurement of impacts.
The mission confirms the relevance of pursuing the experience under a second phase of PROMER, especially in the current project zone, which is justified with regard to: i) the relevance of the overall objective of the intervention, ii) the first positive impacts, iii) the overly limited duration of the current experiment, the achievements of which are still to be consolidated; the core current activities of PROMER are still in the field, iv) the need to define and put in place a sustainable coaching service for rural micro-enterprises, and v) the unsuitability of the financial tools, which excluded a number of potentially viable parties.
Pursuing this experience should maintain an overall developed strategy that targets the training/coaching (advisory support)/credit model at the location of the rural micro-enterprises and with externalization of the financing function. The approach to be used with the existing groups or enterprises should not be exclusive regarding the various types of enterprises to be supported whether in terms of sectors of activity or of size. It should be differentiated according to the type of enterprise: single-person micro-enterprise, artisanal enterprise or enterprise with a vocation and a potential to evolve towards the small and medium-sized enterprise. The mission estimates that the second phase interventions should not be aimed exclusively at the small- and medium-sized enterprises at the risk of directly or indirectly excluding, the poor categories from the communities involved. The actions to be put in place should, in parallel with the targeted support for the rural micro-enterprises involved at a given level of a cluster, further the extension of the clusters involved, particularly in number and diversity of players involved in the cluster or contiguous interdependent clusters.
On the matter of location of the interventions, the achievements of the project and other interventions in Senegal tended to give priority to a pragmatic approach that should not oppose a village and département seat location. The geographic positioning of the intervention in a region should be decided on the basis of the combination of a certain number of minimum criteria: availability of raw materials, accessibility, actual or potential presence of financial services and outlook for a cost-effective advisory support service due to the existence of a potential for developing economic activities.
During a second phase, the extension zones for PROMER activities should correspond to the intervention zones of other projects funded by IFAD in view of maximizing complementarities and synergies requiring effective coordination of the interventions. However, the PROMER experience is still new, the various methodological tools still need to be validated, and so extension of the project intervention zone should be done cautiously. An extension of the project to the entire national territory would seem premature.
The question of financing is central in the problem of emergence and consolidation of the rural micro-enterprises, so it seems appropriate and important that PROMER be able to integrate a specific component for questions related to access to financing for the target public. Nonetheless, although the vocation of the second phase of PROMER is to intervene in zones covered by the other projects funded by IFAD, the mission recommends that the second phase of PROMER be able to integrate a technical assistance service for the projects on the issue of financing. This service could be offered by an internal cell with the required competencies but the option of contracting with a specialized outside organization should be given priority. The intervention of international technical assistance could complete the scheme.
Creation of a sustainable advisory service for the rural micro-enterprise should be one of the objectives of the second phase. The process of withdrawal and empowerment of this service should be provided for from the beginning of project design and planned for in the time allotted. Right from the start, the principle of progress billing for services, particularly for the commercial and training functions, which the entrepreneurs seem to be ready to take over when the services are quality and the positive impacts are clear, should be adopted in a modular manner. Then the question of takeover of all costs for these coaching services cannot be exclusively borne by the beneficiaries but must also be considered from the perspective of subsidy and duration, especially considering the contributions from the State and the local collectives.
Lastly, from the start it would be appropriate to build a relevant monitoring and evaluation and indicators system taking into consideration all of the characteristics of the rural micro-enterprise. The annual audits on accounting, financial and procedural should also be able to intervene in the internal information system.
1/ The interim evaluation mission members were Ms Corinne Riquet, economist, mission leader, Mr. Bertrand Guibert, agronomist, and Mr. Nguala Luzietoso, economist. Messrs. Bakhayokho and Kane, technical advisors to the Minister of Agriculture and Hydraulics, also participated in this mission and represented the Minister therein. Mr. Bakhayokho also accompanied the mission as Chair of the PROMER steering committee. The PROMER managers also accompanied the mission and enriched the discussions and many exchanges on an ongoing basis.
4/ Regional cells intervene in geographic units called ZAEPs (multipurpose economic outreach zones) covering five to six rural communities. Each ZAEP is advised by an AEP in direct contact with the micro-entrepreneurs.
9/ These risk funds fed by a project allocation, with lost equity, cover 4.5% of the credit put in place (or 30% of the risk, estimated at 15% of the total credit actually granted) (70% are chargeable to the institution), or projected a reimbursement rate of 85%).
Special Programme Soil and Water Conservation and Agro-forestry in the Central Plateau (2004)
The Special Programme for Soil and Water Conservation and Agroforestry in the Central Plateau of Burkina Faso commenced in October 1988 as one of the most representative and most ambitious investment projects in IFAD's Special Programme for Africa. Phase 1 covered four provinces (Passoré, Yatenga, Bam and Sanmatenga). A second phase was approved in December 1994 and began in May 1996, expanding the programme area to Boulkiemdé, Sanguié and Namentanga provinces.
The total cost of the programme over its 15-year implementation period is USD 38.1 million, of which USD 26.7 million represents IFAD lending (SRS 011-BF in Phase 1, 80% disbursed; and SRS 044-BF and 0369-BF in Phase 2, 99% disbursed). The second phase was cofinanced by the West African Development Bank (BOAD) in the amount of USD 1.5 million. BOAD also provided programme supervision. Projected completion for Phase 2 is the end of June 2003, following a one-year extension.
In view of the foregoing, and given the possibility of further action in the area, IFAD's Office of Evaluation was asked to conduct an interim evaluation of the programme2. The mission's objective was to assess, in a spirit of collaboration and partnership, the relevance of the actions and approaches taken, measure their impact and any changes made, and draw lessons for the future as to what ought to be pursued, ended or improved.
The interim evaluation focused on three major thrusts of programme intervention: (i) conserving productive resources and securing agricultural production; (ii) rural financing (credits and subsidies for income-generating activities and equipment); and (iii) building local capacity.
The mission worked in the programme area from 24 January to 20 February 2003. The mission members were three consultants and three officials from the regulatory authority. They began with a facilitated self-evaluation by the programme team, visits to the field, meetings with stakeholders and others, and a workshop in Yako to allow for reflection and an exchange of views on the mission's preliminary findings.
Programme context and design
The central plateau covers one quarter of Burkina Faso's surface area and holds 43% of its population. With GNP per capita of less than USD 240, Burkina Faso belongs to the group of least developed countries. Agriculture is the source of livelihood for 92% of the population. The central plateau has been and continues to be one of the most disadvantaged regions of the country owing to its poor soil resources (exacerbated by serious erosion) and high population density (from 70 inhabitants/km² to more than 100 in some provinces).
The programme's overall objectives were, on the one hand, conserving natural resources and, on the other, effecting lasting improvements in production, income and living standards for the population concerned. The target group was composed mainly of rural households farming on less than three hectares of land, or approximately 40 000 smallholdings (20% of them headed by women), representing a population estimated at between 320 000 and 400 000 people living in the 27 departments selected within the seven provinces of the programme area. The programme was to benefit more than 4 000 young people in addition to 1 000 women through its women's advancement and rural credit components.
The main quantitative objectives of Phase 1 were soil and water conservation (SWC) treatment of 28 000 ha of collectively farmed enclosed fields and 10 250 ha of individually farmed outfield plots. The creation of 160 nurseries under the agroforestry (AGF) component was to reach at least 10% of the area covered by SWC measures, i.e. 2 800 hectares. Twenty per cent of these same areas were to undergo crop intensification using various technology packages; 2 643 tonnes of Burkina phosphate and 470 tonnes of NPK fertilizer were to be distributed under this exercise. The research and development component was to address the need for innovation in the spheres of crop intensification and agroforestry, with pre-determined subject matter. A village development fund was to promote the mainstreaming of women in the development process through local savings and credit associations, as well as finance other agriculture-related activities to generate income for the rest of the target population.
Phase 2 called for SWC treatment of 27 500 ha of collectively-farmed land and 30 000 ha of individually-farmed land, as well as the construction of 780 irrigation dikes. Sixty new nurseries were to supply plants for AGF activities on 40 000 ha. Crop intensification activities were intended to restore soil fertility, promote better soil use and integrate crop and livestock activities. Twenty-four groupings of livestock breeders (from eight villages) were to conduct a pilot operation in semi-stabling and rangeland management. The promotion of women's activities, working through 300 women's groups, called for the provision of 900 wheelbarrows, 50 grain mills, the introduction of improved cookstoves, literacy activities and support for income-generating activities by means of microcredit. Training was to be provided to 27 000 people in an effort to build the capacity of farmers' organizations. Agricultural credit was to be used to promote the use of farm equipment, providing partial guarantees to enable the financial institution, the National Agricultural Credit Bank (CNCA), to finance 1 500 carts. An improvement in the institutional capacity of partner service agencies was included but not precisely quantified. Finally, Phase 2 called for the construction or rehabilitation of 180 wells, accompanied by orientation on hygiene and infrastructure management and maintenance.
Implementation and subsequent developments
The organization and management strategy for Phase 1 was based on massive voluntary participation in carrying out the work by the beneficiaries on a contractual basis, in order to: (i) minimize recurring costs; (ii) maximize the use of local resources and competencies; and (iii) lead to self-management by the programme beneficiaries in the medium term. The programme was to be carried out in collaboration with government institutions (CNCA, provincial agricultural research and producer support services). Phase 2 continued on the same basis. The project management unit provided overall direction and financial back-up through technical, administrative and financial management for the programme.
Burkina Faso has undergone far-reaching institutional changes since 1987 as a result of: (i) two major reforms in the Ministry of Agriculture; (ii) implementation in 1992 of an agricultural sector adjustment policy that placed a hiring freeze on all public technical services; (iii) passage of Law 14/99 setting up a procedure to establish farmers' organizations by subsector; (iv) creation by interministerial decree in 2000 of the village land management committees, introducing an important new actor in the process of decentralization, local development and local land management; (v) creation of the province of Zondoma within Yatenga Province (Law 9/96/ADP of 24 April 1996); and (vi) in the economic sphere, the devaluation of the CFA franc in 1994.
The programme design has changed very little overall over the course of the two phases, with both the major objectives and the modus operandi remaining the same since 1987.
The seven provinces within the programme area contain 80 departments and 1 710 villages. By the end of the second phase, the programme had been active in 27 departments and a total of 459 villages (75% of the villages in the departments selected and 27% of all the villages in the seven provinces).
Conserving productive resources and securing agricultural production
In the area of soil and water conservation (SWC), programme documentation indicates that, since 1987, activities involving stone bunding have been carried out on close to 89 600 ha (60% collectively-farmed land and 40% individually-farmed land), an area approximately equal to the objective set. In addition, the programme has built 748 irrigation dikes, improved the traditional zaï system on 32 500 ha, and introduced demi-lune conservation structures on 324 ha. The mission cannot however judge the reliability of the database. The plans for the treated land are not geo-referenced to the site level. The condition and use of land prior to treatment are not specified, and the translation to "treated area" of linear structures such as stone bunding installed on unsurveyed land is not easily verifiable. Still, the technical quality of what has been achieved appears to be satisfactory overall. The implementation process was based on public technical services such as the regional departments of agriculture, water and wind resources (DRAHRH), the programme's outreach teams and village associations. Implementation remained linear through both phases, with no real increase in responsibility on the part of villagers (farmers carrying out the work). In latter years this approach felt the effects of staff cuts at the regional departments (50% of field staff on average as of this writing). The average cost of one hectare receiving SWC treatment ranges from USD 82 to USD 3023 depending on the method of calculation.
In the area of agroforestry (AGF) over the same period, the results posted by the programme are as follows: production of 2.6 million plants (90% of the target), installation of 182 nurseries (107%), assisted natural regeneration (ANR) on 17 058 ha (79%), organization of 45 groups of livestock breeders in 15 villages, implementation of a pilot livestock operation in semi-stabling as well as plantings of ligneous and herbaceous vegetation along stone bunding. Technically speaking, some of these results appear promising and of interest, especially the ANR procedure and the pilot livestock project. On the other hand, the approach taken is not sustainable for the planting and nursery component (a collective, non-economic approach) and not well adapted (the final ownership of the plantations and profits generated by them is not clear to farmers). Stakeholder involvement in plantations is also observed to be low. The survival rate after three years is not encouraging (around 30%) compared to the amount of effort and cost involved.
In the area of crop intensification, since 1987 the programme has installed 29 000 compost pits (112%), distributed 2 936 tons of Burkina phosphate and trained 21 producers of improved seed for total yield of 13.1 tonnes (maize, millet and sorghum).The composting technique was not appropriate to the constraints and capacities of the rural environment in terms of the availability of water and raw materials and the difficulty of work, and compost production volumes are far from proportional to the number of pits dug. In addition, the approach used to distribute the improved seed does not allow for determining the dissemination rate in the rural environment or ensuring the permanence of the system.
In the area of research and development, the national research institute INERA has produced 16 technical fiches on SWC, AGF and crop intensification that describe the various techniques and quantify the results obtained, but the economic and social data remain to be developed. Certain assumptions need to be better adapted to the farmers' logic, strategy and resources (as in the case of compost production) if a more satisfactory acceptance rate is to be achieved.
In the area of village water management, the programme has installed or rehabilitated 201 functional wells using an efficient procedure; 60 new wells are to be built under a contract that is to be executed by programme completion. The benefits of this type of intervention are well known and need not be demonstrated.
Financing of rural activities
Medium-term equipment credits. Overall, 1 036 credits have been extended at a rate of 9% per annum, exclusively for the purchase of carts, totalling 194 million CFA francs (two thirds of the target amount): 351 village associations have benefited from this operation in eight provinces, and approximately 16% of the credits were made to women. The repayment rate is reported to be 87%. Difficulties have arisen in the partnership with the Agricultural and Commercial Bank (BACB, formerly CNCA), which has in some cases taken a very long time to review applications and issue financing (up to 13 months); nor has it fulfilled its commitments in terms of credit recoveries since 2001.
Short-term credits for income-generating activities. A total of 1.09 billion CFA francs in credits has been granted, 64% to women. These are mainly small business credits (87%) and start-up credits (9%). A total of 292 local savings and loan organizations have been created and/or assisted (with a total of 8 338 members, of whom 76% are women), including 249 village banks belonging to the national federation of credit unions (RCPB). Half of the villages in the programme area have benefited from the establishment of these 292 local savings and loan organizations, but not all of them were established by the programme (some pre-existed it). The penetration rate of these organizations remains quite low (on the order of 4%).
Subsidies for women's advancement and soil and water conservation activities (subsidies for equipment). A total of 688 carts and 1 197 wheelbarrows have been distributed through partial or full subsidies. Although the wheelbarrows were originally intended exclusively for women, the carts were to be distributed to both men and women. In fact, they were distributed exclusively to women as well. Virtually no equipment has been used for SWC activities as planned. In almost all cases, it has been used for hire.
Building local capacity
The pilot approach to land management that was implemented in 8 of the 15 villages targeted, led - following a participatory, informational and educational procedure - to the elaboration of village development plans. Despite the request made at the time of the mid-term review in 1999, no funds were made available to implement the village development plans. The programme did train leaders in fund-raising, with the expected difficulties in making use of such training (remote locations, illiteracy, lack of negotiating experience). Therefore it is difficult, even after seven years, to reach any conclusion on this component, except for the preparatory phase which, although long (2.5 years), seems to have been assimilated by the villagers.
Support for farmers' organizations. The main results have been two identification surveys conducted (1 264 village associations selected), a typology of the latter (93% are non-functional, i.e. not engaged in any real activity), training of 21 684 producers, including 7 316 women, in matters such as cooperative management, management of collective equipment and economic units, planning and programming of activities, mobilizing local resources, and so on. In addition, 362 officers from partner services (DPA) have received similar training, and assistance was provided for the restructuring of farmers' organizations and their legal recognition in accordance with Law 14/99.
Training. The programme, in Phase 2, provided 264 000 person-days of training on 37 subjects to producers, support services and PMU staff, including 30 000 person-days on crop intensification and techniques and more than 21 000 person-days on the operation of farmers' organizations. Specialized training was provided to 324 professional herders, 256 nursery workers and 19 well-repair workers; 8 265 women were trained in improved cookstove techniques and 1 129 women received literacy training.
Institution-building. The principal results are as follows: (i) the establishment of 17 private and public-sector agreements and seven outreach teams in Phases 1 and 2; and (ii) elimination of annual contracts with villages; (iii) active participation in the implementation of eight technical provincial consultation meetings and the holding of seven annual review and programming workshops with all programme stakeholders.
Impact, relevance and efficiency of the programme
The evaluation confirmed that the programme had many positive effects. Among the main ones are an improvement in productive capital (land) and crop production: SWC interventions on some 90 000 ha4 (including 5 000 ha of reclaimed formerly non-productive land) and crop intensification measures led to an average increase in cereal yields on the order of 25% (from 400 to 500 kg/ha on average) on 20 to 30% of cultivated land in 459 villages. An estimated 50% of the land cultivated by the 34 500 households and association members covered by the programme received treatment, for an average of 2 ha per household. In addition, about 5 000 households applied treatment to an additional 20 000 ha of ‘non-programme' land, following the example of neighbouring fields. Overall, an estimated one half of all households in the villages within the programme area have benefited either directly or indirectly. In the programme area as a whole, an estimated 7 000 to 9 000 tonnes of millet and sorghum have been added to cereal production thanks to the programme. Many farms have improved their equipment as well (in particular hand tools and carts).
Access to local credit for the 8 300 members of local savings and credit associations has led to a significant rise in income generated by small business and livestock activities. Overall, an estimated 815 million CFA francs in profits have been generated thanks to microcredits granted.
Food security in households having improved their land resources increased, benefiting at least 350 000 people. Average annual cereal production per farm reportedly increased from 1 600 to 1 800 kg of millet/sorghum, i.e. from 80% to 90% coverage of annual cereal needs for an average household (estimated at 2 tonnes per year for a household of 10). These average figures naturally break down quite differently in actual households. In particular, the technical solutions put forward by the programme (stone bunding, traditional zaï techniques, organic compost) are all very labour-intensive. Those households with a high ratio of consumers to assets and unable to obtain outside labour undoubtedly remain at a disadvantage.
The effects of SWC interventions on agricultural production are much more noticeable in years of low rainfall, since they reduce vulnerability to climatic fluctuations. Before attributing the general improvement in food production and food security in the area concerned to the programme, we should take into account the fact that average rainfall has been higher in recent years than during the initial programme.
Some 233 000 people in 204 villages have benefited from improved access to drinking water and therefore improved health conditions (especially for children) thanks to the installation of 261 wells. Women have clearly benefited from this village water component as well as other measures taken to relieve their workload (transportation equipment in particular) and enable them to engage in revenue-generating activities. Their position within village society has improved thanks to their economic weight (they generally represent a healthy majority in local savings and credit organizations) and the new knowledge they have acquired.
However, the expected impact from building social capital and local and collective capacities did not materialize in proportion to the investment made. With respect to SWC activities, the programme implementation strategy essentially depended on state technical services and outreach teams, with villagers taking on little responsibility and contributing mainly manual labour. The changes in public services in the past 10 years (SAP of 1992) at the regional and provincial levels clearly make this an outdated approach. The programme was unable to prepare villages and village land development committees to take on responsibility for managing their own land resources, nor did it stimulate the emergence of private operators (with the exception of some technical training for nursery operators and well-repair workers). This was clearly a major error in design and implementation of the second phase of the programme, and had an adverse impact on prospects for a sustainable development process.
The programme's contribution to certain positive changes – such as the overall improvement in plant cover, a slowing in the rural exodus and the sharp decrease over the last 20 years in the proportion of households living below the poverty line – is difficult to assess with any certainty. Many exogenous factors, such as changes in rainfall, the rise in gold washing activity, and opportunities for emigration and employment in Côte d'Ivoire, have all played an important part. Also, although 35 000 to 40 000 farms have benefited from the programme interventions, the proportion of the target group reached, i.e. the poorest and most disadvantaged households, cannot be determined using the approaches applied.
The programme has made it possible to validate or confirm certain techniques in SWC, agroforestry (ANR), livestock raising, village water management and microcredit, which can today be replicated on a larger scale. On the other hand, little progress was made in the areas of fertilization, crop intensification and reforestation, all of which are essential in a context of crop settling. The research and development component did not contribute any major innovations to address farmers' expectations but did however refine already existing techniques and verify their quantitative results. Little progress has been made on evaluating the economic and financial returns on these techniques or analysing their relevance to different types of farming based on the availability of labour, conditions of access to land, and so on.
The programme has once more illustrated the importance and potential of local credit and the excellent response of populations in using such credit to engage in income-generating activities (averaging 75% net profits after repayment). What is needed at this point is an expansion and consolidation of rural financial services networks, and an increase in the still-low penetration rate. Also, institution-building through the components of support for farmers' organizations, village land management committees and public technical services did not meet expectations and will require an in-depth revision of approach. The local land management (gestion de terroirs) approach was not effective either in putting in place overall management plans contractually at the village level or in reflecting on and implementing rules for managing natural resources within communities.
The assessment of the programme results in the main domains of impact is summarized in the table below:
Domains of impact
Rating of Impact
|1. Physical and financial Assets (principally land and agricultural equipment)|| |
|2. Human Assets (principally improvement in access to potable water and training)|| |
| 3. Social Capital and People's Empowerment |
(strengthening of groups and management unit)
|4. Food and economic security (reduction of food-deficit and food risk)|| |
|5. Environment (reduction of erosion and retention of waters)|| |
|6. Institutions, policies and regulatory framework|| |
(4= high; 3= substantial ; 2= modest ; 1= négligible)
From the point of view of methodology, the approach planned at the outset was pertinent (a contractual transfer of responsibility in response to organized village demand through permanent local providers) but was not implemented in the spirit of the appraisal report. A systematic approach of working with groups was not the most appropriate in certain areas (crop intensification, reforestation and agroforestry) and was not able to provide assurances of reaching the target population (only 30% of the population was represented by these village associations). The SWC and AGF approaches suffered from a lack of reflection and support for implementation strategies in order to achieve goals, as well as conditions for sustainability of the dynamics used. The programme concentrated its attention and energy on physical achievement of quantitative objectives and paid only limited attention to social and institutional conditions and to ensuring the permanence of the development process. What is needed is a better definition of the development objectives of programmes, based on a more accurate and more qualitatively-oriented logical framework and –once again – putting in place monitoring and evaluation mechanisms that can (assuming adequate resources) not only monitor compliance but also evaluate the timeliness of actions and relevance of methods with respect to the strategic objectives and the philosophy of the contractual documents. Also needed is better capacity on the part of decision-makers (project management, oversight ministry and providers of funding) to fine-tune a programme in the course of execution. Certain pertinent recommendations were made by external evaluations in 1993, and then again in 1997, but were never taken into account. In this essential area of ‘strategic direction' and evaluation in the course of implementation, supervision by the cooperating institution has once again shown its limitations.
The institutional arrangements and partnerships have been shown to be difficult to apply at times in a constantly changing environment. Having virtually all of the programme contingent on public technical services very soon showed itself to be an ephemeral approach. It would be advisable for future plans to leave open the possibility of distributing tasks differently (advisory assistance, training, implementation, monitoring) according to the type of provider (public, private) and not to hesitate to favour the emergence of private operators if the opportunity arises. An approach that provides for flexible assistance that is gradually phased out should be planned from the outset. A prior audit of institutions operating in the field should be carried out before any project is identified, to reach a better understanding of the competencies, resources and capacities of local operators as well as their objectives, strategies and costs as providers.
The experience of this programme also shows the importance of having a regional and local strategy based on a technical targeting process. It is not possible today to determine the significance of the 90 000 ha of treated land vis-à-vis the size of the problem in the area, or where emergency regions remain to be dealt with, or where there is land with good potential for supplying stone. A spatial targeting strategy based on constraints and potential does not rule out a sound partnership that can be formalized in contracts with the villages concerned. The overall watershed approach warrants a review to better build in protection for surface water plans (e.g. silting of the Bam, Yako and Ouahigouya reservoirs). Also, it is difficult to evaluate actions already concluded and programme investments for the future on a large scale unless it is possible to determine and measure, on a map and on each plot of land, the baseline situation, programme achievements, and what remains to be done. Simple ways of achieving this by using survey photographs and GPS could provide ad hoc support for this type of action.
The programme evaluation reveals the need to move systematically toward an economic, financial and commercial approach to the proposed actions. It is important that each participant be able to place a cost on investments and activities and measure their potential returns, whether with respect to soil and water conservation, compost production, improved seed or reforestation activities. It is quite regrettable that after 15 years of interventions the economic and financial returns on the proposed investments are not better defined, since the ‘transferability' of the functions and estimates of necessary and sufficient public subsidy levels depend on such a definition. Programme management, as well as partners providing support and investigators, must adopt a new attitude in this sense. The ‘beneficiaries' already possess this mentality and express it more or less intuitively through the intensity of their response to the actions proposed.
The evaluation has once again demonstrated the need for reliable baseline studies and a better knowledge of the constraints but also the strategies and practices of the target population. This information, coupled with a pragmatism that is currently lacking in the research, would result in proposals of actions more in tune with the environment and facilitate their adoption by the target group.
It is important to maintain and strengthen consultative agreements between projects and between public actors and civil society (farmers' organizations, NGOs, etc.) to ensure harmonization of interventions in the same villages in related subject areas (and/or distribute them better geographically).
This programme also shows the importance of formalizing the programme commitments with the target population, and the importance of having local stakeholders gradually take on maximum responsibility (for logistical, administrative, and then financial aspects) and phasing out external assistance over time. In parallel, staffing ratios should be reviewed and a strategy adopted to gradually increase the number of villages involved.
Addressing the issues relating to soil reclamation and conservation in the central plateau is a race against time. In 15 years, the population has grown from 2 to 3 million inhabitants. It has not been possible over the same period of time to treat more than 20% to 30% of the surface area, and the process will tail off significantly during the post-project period. The lessons learned from this experience should enable programmes to be developed to accelerate the dynamics of development activities built much more upon sustainable local initiatives (that generate income if possible) that take on a management role and depend less on project institutions. Much is at stake. The central plateau now holds more than half the country's population.
Based on the lessons learned over the course of this programme and the size of the environmental, agricultural and human challenge at hand, the interim evaluation mission recommends the following:
Preparation of a new sector intervention in the area of agricultural land management issues (SWC, AGF, surface water and small-scale irrigation) and crop settling (fertility management), following prior conduct of in-depth studies which are indispensable for any new project, namely: (i) a zoning study to measure the development of and current trends in water and wind erosion in the central plateau; (ii) a study of the situation and capacity of current local operators and providers; and (iii) implementation of pilot mini-operations to determine the feasibility of new rural activities (groups of young people engaged in land conservation, for example).
Consideration of the lessons learned during this programme in order to commit to a fundamentally different approach, establishing from the outset an evolving strategy based on local actors (farmers' organizations, SMEs and local pieceworkers, village land development committees) and a planned phase-out of the project structure. Under this approach, the project could call for a support fund for agricultural land development to be set up as a starting point, to be managed directly by village land management committees and/or agencies and accompanied by capacity-building and research-action measures.
Establishment of a programme to develop financial services (microcredit and others) within the strategy of local financial institutions but with more ambitious, better distributed penetration rates (just 4% currently after 15 years of intervention).
1. The interim evaluation mission was made up of Mr Hubert Boirard, mission leader and rural institutions expert; Mr Bertrand Guibert, agricultural engineer, Ms Corinne Riquet, economist specialized in microfinance, and three officials from the line ministries.
2. The programme had been the subject of a mid-term evaluation during the first phase (IFAD, 1993), a case study as part of the evaluation of the Special Programme for Africa (IFAD, 1997) and a study of the impact of the SWC, agroforestry and crop intensification components during the second phase (IFAD, 2001).
4. The evaluation mission was not able to verify the total land surface improved under the programme. The positive impact on beneficiary villages and households is clear, but only a large-scale survey could confirm the actual extent of the impact at the overall programme level.