Interim evaluation
With an estimated GNP per capita of just USD 100, Ethiopia is one
of the poorest countries in the world - about one third of the population
live in absolute poverty. Ninety per cent of the population, which was
60 million in 1995, live in the rural areas, where subsistence agriculture
is the main economic activity. Because of higher rainfall and healthier
climatic conditions in the highlands (above 1500 m asl), nearly 90% of
the population choose to live there. The Special Country Programme area
lies in the crescent shaped Ethiopian highlands east of the rift valley,
and was defined as being between the 400 and 800 mm isohyets. The area
consists of both extensive plains and also a dissected ridge section stretching
to the Somali border; temperatures and rainfall vary with altitude and
geographic location, with the higher massif receiving over 1,000 mm of
rainfall. The main crops are wheat, barley and maize, plus various pulses,
fruits and vegetables, but yields at the time of project design were generally
low. About 10,000 ha in the area was estimated as being in farmer-developed
small-scale irrigation schemes (SSIs), which usually consist of simple
diversion structures, earth canals and poorly or completely unlevelled
fields, resulting in very low water use efficiencies. Many rural areas
have no roads at all, and the transport of agricultural produce relies
on animal transport over mountain trails.
Project design and objectives
In 1984, following an extensive and prolonged period of drought, IFAD
prepared the Rehabilitation Programme for Drought Affected Areas, which
was designed to provide immediate relief in the form of seeds, tools,
etc. and to rehabilitate drinking water and primary health care facilities.
This was a three year (USD 19.4 mn) programme financed with the BSF. In
addition to the major components, there were also small components forsoil and water conservation andsmall-scale irrigation schemes;
during appraisal of the Rehabilitation Programme IFAD agreed to extend
these activities in a new programme, to be financed under SPA.
Target group
The programme’s target group consisted of about 25,000 farming
families in drought prone areas who were expected to benefit from the
six-year development and rehabilitation of 6,100 ha of SSIs, and a further
300,000 farm families who would ultimately benefit from the soil conservation
measures. The creation of small vegetable gardens and the funding of income-generating
activities would also bring benefits to about four hundred women. In a
normal rainfall year, it was estimated that incomes from farm and off-farm
sources amounted to about USD 65 per capita, hence it was assumed that
most highland farmers were living in absolute poverty.
Objectives and components
The main objective of the Small Scale Irrigation programme was to
achieve increases in crop production and hence to improve food security
in drought prone areas, with the intention of providing a minimum food
supply in deficit years and a marketable surplus in more normal years.
The Conservation and Agricultural Support (CAS) programme addressed
the problems of land degradation in the highlands, with the specific aims
of fully involving the farming communities in the design, construction
and maintenance of conservation schemes, and the promotion of activities
which would be of demonstrable long term benefit to the participants.
Each programme component consisted of a number of related sub-components;
total base costs were estimated at USD 29.5 mn, to which contingencies
added a further USD 10.7 mn. In summary the base costs covered:
| Small Scale Irrigation Prog. (national
support, zonal support, agricultural support, scheme investments) |
USD 20.1 mn (68%) |
| Conservation and Agricultural Support
Prog. (conservation planning, trials, ag. extension, bund stabilisation,
women’s support) |
USD 8.3 mn (28%) |
| M & E |
USD 1.1 mn (4%) |
Under the SSI programme there were four sub-components for: (i)National
Support, which concerned the strengthening of the Irrigation Development
Department (IDD) in the MOA to enable it to prepare a national strategy
and policy to provide the framework within which the programmes could
be implemented, and was also to facilitate the preparation of criteria
for scheme selection (a total of 5 years of TA, plus equipment and vehicles,
additional staff and training was included in this package); (ii)Zonal
Support, the zonal offices of the MOA/IDD would be upgraded to enable
them to actually implement programme activities, this would include buildings,
construction equipment, vehicles etc.; (iii)Agricultural Support,
would be provided to enhance the capabilities of MOA in irrigated agriculture,
this would involve training for the extension agents and research at the
Soil and Water Management Research Centre; (iv)Scheme Investments,
would fund the rehabilitation and also construction of SSIs ranging from
a few hectares to a maximum of 200 ha, by providing for construction materials,
operation of construction equipment, skilled labour etc.- unskilled labour
would be provided by the beneficiaries. Under the construction programme
the requirements for roads would be estimated and construction undertaken
by unskilled labour on the basis of food-for-work, financed by WFP.
The CAS programme would provide: (i)National Support for Conservation
Planning, in the Community Forest and Soil Conservation Development
Department (CFSCDD) to enable it to prepare a National strategy, to become
the centre for conservation planning and to co-ordinate procedures with
other government departments and with donors; (ii)Zonal Support/Conservation
Trials, six trials sites of 15-20 ha each were to be identified, and
30 on-farm demonstration trials of about 5 ha each were to be established.
Under this heading,Agricultural Extension would be provided, closely
linked to the trials in order to develop extension messages and ensure
that farmers were fully aware of the potential of the conservation activities,Bund Stabilisation, was to be encouraged by planting suitable cover
and binding crops, in the first instance a number of nurseries were to
be established andRural Women’s Development Support would
be included, aimed at promoting the establishment of vegetable gardens
and creating a development fund which could be used by women for small-scale
income-generating activities.
Monitoring & Evaluation was to be undertaken by the existing
units in the MOA and the Agricultural and Industrial Development Bank
(AIDB), but both would be assisted by the programme, including the provision
of office equipment, vehicles, training, TA and incremental operating
costs.
Expected effects
and assumptions
It was estimated at appraisal that 2,750 ha would be rehabilitated
and a further 3,350 ha would be developed in new irrigation schemes; at
full development (12 years) yield levels under irrigation were expected
to have doubled. Agricultural extension advice would cover the whole farming
area, i.e. including rainfed crops; conservation activities were anticipated
to gradually improve practices and production levels, and yields were
anticipated to rise by 20% in the rainfed areas over the six year programme
period. These increases should allow income levels to rise to between
USD 109-167 per capita. One aspect of the conservation programme, bund
protection, was expected to affect between 50-100,000 families, covering
9,500 ha; the treatment would reduce maintenance requirements on bunds
and increase the fodder available for livestock. The support at national
level was expected to produce consistent policies with respect to irrigation
and conservation activities.
The programme contained a number of assumptions, these were: (i) that
the main benefit from the irrigation programme would be increased cereal
production; (ii) that a high priority would be given to soil conservation
activities by the government; (iii) that national policies on irrigation
or conservation would be widely disseminated and practised; and (iv) that
farmers would accept and adopt maintenance activities for the irrigation
facilities provided.
Evaluation
The multi-disciplinary interim evaluation mission visited Ethiopia
in November 1995. There was no impact data available to the mission, but
farmer interviews were held whenever a site was visited, as well as meetings
with women’s groups and water user associations (this resulted in
contacts with about 50 beneficiaries). Discussions were also held with
government Departments and agencies, programme officers and other donors,
and a wide selection of documents made available to the mission was reviewed.
Implementation context
In 1990/91, security in the programme area deteriorated and implementation
was adversely affected by the events leading up to the violent change
of government in 1991, by the administrative hiatus that immediately followed
those events, and by subsequent decentralisation and associated restructuring
that has occurred with irregular frequency. From the beginning of the
project, co-ordination was merged into the complex system of committees
(national, regional, zonal, and programme) that was trying to manage agricultural
development in Ethiopia. Consequently, in the programme’s opening
years, when lines of communication were being established between the
implementing bodies (i.e. zonal offices of central government departments)
and the centre, few programme activities were begun.
In 1992, the transitional government introduced decentralisation,
in which many of the activities of central government were transferred
to new ethnic regions and urban councils. Many central government staff
working on the programme were transferred to regions and ceased to be
involved in programme implementation, the system of central co-ordinating
committees collapsed, and programme implementation became the responsibility
of two of the new regional offices, two urban councils and appropriate
subordinate zonal offices. Programme assets, once held centrally, were
divided between regions and zones, and in some cases lost to the programme
altogether. Restructuring by central government resulted in the separation
of the regional and zonal departments dealing with irrigation and agriculture,
and occasionally that separation was manifested as mutual hostility between
the departments and non-co-operation in programme activities. In a later
attempt to revive flagging agricultural activity, the government allowed
the unpopular peasant collective farms (producer co-operatives) to collapse/,
which had severe implications for implementation. By 1991/92, the fifth
year of operations, all aspects of the programme were seriously behind
schedule. However, activities were then restarted in earnest, the irrigation
and supplementary conservation targets were reduced, and the programme
was extended for a further two years, to December 1995.
Project achievements
All significant achievements in the programme’s two main components
trace their start to either 1992 or 1993. That applies particularly to
staff foreign training, when two thirds of the programme’s USD 1.4
million budget was expended. However, it was to be of little avail to
programme performance; about three quarters of the 35 staff who attained
higher degrees abroad had left the service of the programme, and sometimes
of government, by December 1995.
By July 1995, eight years after the loan became operational, and
after two extensions, loan and grant disbursements amounted to USD 13
million (38.5%). About 1,300 ha of incremental irrigation had been
established (which, with an existing 200 ha, meant there were 1,500 ha
of small-scale irrigation schemes), but other than trial and demonstration
sites, none of the mountain areas subject to erosion had benefited from
supplementary soil-conservation techniques. About 5 000 farm families
had benefited from the irrigation schemes, and over 300 women had established
gardens. Very few women had benefited from assistance with income-generating
activities.
The absence of a government policy on SSIs means that there were
no clear guidelines on either cost recovery from beneficiaries, or the
necessary extent of participation and commitment by those beneficiaries.
Either before or on completion of their scheme, beneficiaries are asked,
through their water users’ association, to agree to contribute towards
capital and operation and maintenance costs. Because of their vagueness
and open-endedness, these agreements are unenforceable, and rarely result
in beneficiaries making any contribution towards scheme costs. Of the
25 schemes completed at the time of the IE, one has started making token
cost recovery payments (although such payments are already in arrears).
These are sufficient to cover about one fourth of normal operation and
maintenance costs, but make no contribution at all to capital costs. The
positive achievements of small-scale irrigation are at risk in the medium-
to long-term because of the absence of a clear workable government policy
on cost-recovery and responsibility for scheme operation and maintenance.
Although scheme design has usually been good, the procedure for scheme
selection left much to be desired. Notwithstanding the agreement at appraisal
that government would prepare a definitive set of scheme-selection criteria,
and notwithstanding the presence of a suggested set of criteria in the
appraisal report, such criteria were never formally completed by government.
The consequence is that at least two completed schemes are not in drought-prone
areas, some schemes have ignored the rights of down-stream water users
and other schemes have failed to make provision for compensation of lost
land (to canals, headworks, etc.), or for re-allocation of land (when
farmers give up land in the command area to increase the number of beneficiaries).
None of the schemes is designed to prevent infestation by bilharzia-carrying
snails, and the investment costs of some schemes were so high, that they
were not economically justified.
Technical Assistance was supplied to the irrigation departments in
Addis and in the Zonal offices, however, because of the political circumstances
in Ethiopia, the TA in the Zonal offices could not function effectively
or be widely deployed. The TA cost USD 3.7 mn, or enough to construct
a further 1,300 ha of new irrigation schemes. It would probably have been
wiser to have postponed the technical assistance to a period when zonal
irrigation departments were more active, i.e. from 1993. This issue, and
the question of scheme selection generally, were matters on which the
IDA supervision missions could have been expected to provide some comment,
but they did not appear to rate them as important.
Although three conservation-based agronomic trial sites were established,
one was reduced in size (to 3 ha) and is ineffective. Bund stabilisation
and crop production trials were installed on the two operating sites for
the first time during 1994/95, and yield data were collected from one
of these sites. A creditable 20 (out of 30 planned) on-farm demonstration
trial sites have been set up in the programme area, while only 12 (out
of 100 planned) farmer demonstration plots are in place. Soil conservation
activities did not benefit from TA, because the budget was never restored
after 1991, and specialised staff needed to manage the sites were never
recruited. Vehicles have been in short supply, extension training materials
have not been forthcoming, and funds have never been available when required.
Consequently, although the zonal extension service has used the agronomic
and demonstration trial sites for their own training purposes, it has
been unable to formulate any conservation-based agronomy messages for
farmers. This may account for the absence of bund stabilisation measures
anywhere in the target area.
Soil Conservation and Agricultural Support. The bunds in Harerge
are constructed of rock and soil, and unless they are protected by vegetation,
they will be costly to maintain, and they will not be fully effective
in abating soil erosion. By rehabilitating 12 ha of existing grass/legume-seed
nurseries, the programme acquired a resource that was theoretically capable
of supplying all the programme’s seed requirements. Unfortunately,
the administrative difficulties of the programme’s opening years,
and the violent disturbances of the middle years meant that only 10% of
annual seed requirements were produced. At the time of the first programme
extension in 1992, the target of 9,500 ha of bund stabilisation was
reduced to 1,800 ha, with a proportionately much lower seed requirement,
but so far, even this has not been achieved. However, by late 1995, the
nurseries were planted near to capacity and showed good potential for
fulfilling the revised seed requirements.
Agricultural support features of the programme have been positive.
Irrigation agronomic trials have been successfully established on nine
sites for at least one year, and promising results from those trials have
been tested for a further year. Farmer training appeared to have kept
pace with the rate of irrigation site completion. Despite some shortages
of staff, the extension service has successfully operated a modified training
and visit system, and has achieved extension worker:farmer ratios of around
1:1,300 or better. Crop production on the newly completed irrigated schemes
concentrates on annual vegetable crops and perennial crops rather than
grain. All schemes faced limited markets for their vegetables and downward
pressure on crop prices was already apparent. Vegetable yields were lower
than anticipated at programme appraisal, but provided cropping intensities
of more than 518 are achieved, irrigated vegetable production is still
profitable. Seed shortages, especially of biennial crops, were reportedly
one of the greatest production constraints.
Women’s Development. The successful creation by the programme
of over 300 women’s gardens (between 200 m² and 600 m² each)
has not only improved family nutrition in East and West Harerge, but has
increased women’s income too. However, nocredit has been
provided either for women’s income-generating activities or for farmers
to fund their own irrigation development. Crop marketability is being
affected by poor access; no roads have been constructed under the
programme’s USD 1 mn food-for-work budget.
Effectiveness
of the M & E system.
A Project Coordination and Monitoring Department was established
in the MOA, with a Division in charge of M&E; 12 man-months of TA
were provided for the design of the M&E system and specific performance
indicators were identified and a logical framework prepared. In 1991 all
activities stopped and the M&E Division was included in the general
restructuring which later took place, but now the carefully constructed
functions could no longer apply because they were designed for a centralised
system and not the regional system that was established. When the M&E
system was functioning as designed the zonal officers collected a considerable
amount of detailed information and produced comparisons of progress against
targets. No evaluations were undertaken, supposedly because in the early
years of the programme there were no impacts, but zonal staff also did
not consider that evaluations were part of their duties.
Effects assessment
and sustainability
By the time the programme ends in December 1996, total expenditure
is likely to be USD 16 million of which USD 13.8 million will
have been expended on small-scale irrigation activities (including institutional
strengthening and capacity-building), and USD 1.8 million on
supplementary soil-conservation and women’s activities. About 2,500
ha of small-scale irrigation will have been completed, to benefit about
8 000 farm families. Incremental irrigated crop production is expected
to have an annual value that stabilises at USD 1.9 million, compared
to an anticipated USD 4.4 million at programme appraisal.
Up to December 1995, operation and maintenance costs of completed
irrigation schemes were being paid from programme funds. It is highly
improbable that regional government can take on these costs when the programme
ends, and it is apparent in most cases that beneficiaries are unaware
of the impending responsibility. Without clear commitments that someone
(beneficiaries and/or government) will pay operation and maintenance costs
and major renewal costs when necessary, none of the schemes completed
under the programme is sustainable. In addition, on some of the schemes
there are signs of discord between the upstream and downstream water users,
and between those who have "lost" or gained from the development,
i.e. have not been adequately compensated for land lost to canals, headworks
etc. Also, without a proper cost recovery scheme that meets at least some
of the capital costs, then the development programme cannot continue without
donor support.
Farmer-participation in small-scale irrigation design and construction
has been nominal only, and has been inhibited by farmers’ reservations
about the democratic legitimacy of imposed institutions such as Peasant
Associations (government’s smallest unit of administration) and Service
Cooperatives. Even water users’ associations have usually been formed
only at government urging. True beneficiary participation therefore will
not be a reality until democratic farmer-initiated organisations are established.
Farmers’ and women’s reluctance to form cooperatives is generally
attributed to the stigma of coercion and oppression they acquired during
the Marxist-Leninist era. This is also affecting the credit facility;
the mandate of the bank only permits it to lend to groups that are corporate
legal entities (e.g. cooperatives), and because so far not one women’s
group or water users’ association has transformed itself into a cooperative
under the new cooperative law of 1994. AIDB (now the Development Bank
of Ethiopia) has used virtually nothing of the USD 1 million or the
USD 240 000 credit lines made available by the programme for
small-scale irrigation and women’s income-generating projects respectively.
Traditional irrigation practices do not seem to cause significant
problems of soil erosion - in fact farmers seemed more aware of potential
problems on irrigated areas. Water is not used efficiently, and there
is much over-irrigation, but water quality is good and irrigable land
is reasonably well drained. The use of chemicals and fertilisers is very
low, and is likely to remain so for some time. Overall the programme is
not a cause for concern from the environmental aspects, although it is
a matter of regret that in fact there have not been any more positive
gains from the sub-components in the conservation programme.
There is some evidence that the development of kitchen gardens has
contributed to an improvement in the nutritional standards, and may also
be providing small surpluses for sale.
Recommendations
and lessons learned
Many of the lessons that have emerged from evaluation of the Special
Country Programme are not new. The Staff Appraisal Report was aware of
the need for commitment by government, farmers, and departmental staff
if the programme were to succeed. For example, irrigation scheme selection
criteria were recognized as of paramount importance to the success of
the irrigation component, and were even a feature of the loan agreement.
There was a clear implication in the report that irrigation schemes would
face trouble without the use of proper selection criteria. That the government
did not define the selection criteria is no fault of the Staff Appraisal
Report. Praise is due to its authors for attempting to steer the programme
away from familiar dangers. However, despite this prior knowledge, one
of the primary lessons of this evaluation is that without proper selection
criteria properly applied, irrigation schemes will not be sustainable.
Connected to this, unenforceable cost recovery agreements between
government and farmers, in which the latter agree to contribute something
later in return for something else now, are of little value in promoting
participation, ownership and commitment. The process needs to be turned
around and made enforceable: farmers should be required to contribute
something in advance of any investment of government funds, as their part
of the investment, to engender ownership and commitment.
Regional and national government needs to play a much more effective
role in promoting awareness of cooperative legislation, and they should
make registration of WUAs a condition of public investment in irrigation.
Government should then enter into irrigation support and cost recovery
agreements only with WUAs that are corporate legal entities.
Future project appraisal reports should contain a detailed plan,
prepared by the project formulation or preparation mission in consultation
with the borrower, and endorsed by the appraisal mission, which should
equip the implementers for the tasks that are expected of them, and give
guidelines to which they could refer during implementation. Similarly,
sustainable irrigation development requires that planners (i.e. the engineers)
must be re-oriented towards social, environmental, and economic aspects
of planning. In addition, any future irrigation or other project or programme
should limit itself to a single operational region.
Given the apparent waste of programme funds on overseas academic
training that had no long-term effect, future funding for foreign training
must be carefully considered in the light of the opportunity cost of the
funds involved and the real long-term capacity-building benefits to be
derived. Where the training for the public sector is thought to be worthwhile
(e.g. M&E), then future programmes should take account of post-graduate
salary differences between public and private sectors, and "top-up"
where necessary.
Supervision missions should be geared to what is implied by the term;
they should be more frequent and of longer duration, aimed at keeping
the project on track in qualitative as well as disbursement terms. For
example, supervision should be involved in approval of investment decisions
for sub-projects within a programme.
Farmers' associations need to be democratized, so that they become
genuine people’s organizations where farmers can debate, analyze,
and decide what they need, and learn how to solve their problems. Farmers,
and the civil society they inhabit, should have the power to participate
democratically in the decision-making process.
Women’s groups should be given assistance to organize themselves
into legally recognized organizations, as a means of empowering them to
participate in available credit schemes.
The market’s competitive capacity for designing and building
irrigation schemes should be explored as a matter of course in project
implementation.
The M&E system should be redesigned to conform to the new federal
structure of Ethiopia. Given the increased number of authorities within
the federal system, the redesign should attribute particular care to providing
smooth information flows between the programme’s grassroots and the
central administrative structure. Officers in charge of central, regional
and zonal M&E should be provided with specific training. Particular
efforts are needed to develop impact-evaluation skills. In that connection,
M&E officers in the field should be provided with the financial and
logistical means to keep in frequent contact with target groups.
At an early date, the Minister of Agriculture or a high-powered delegate
should meet with his or her counterpart at the Ministry of Finance to
settle the issue of unnecessary payment of customs duty on equipment and
vehicles imported under the programme’s aegis.
Given the great need for some action that would ameliorate soil erosion
and bund destruction, the donors should propose a new bund stabilization
project for East and West Harerge. This would allow completion of the
SCP bund-stabilization work, formulation of extension messages, and diffusion
to hill farmers. Alternatively, a bund stabilization component could be
included in a potential follow-up project including the SCP project area.
A soil scientist with experience in soil erosion measurement techniques
should be consulted for recommendations on corrections for errors in the
present system of erosion measurement.
