Evaluation of Project Performance and Impact
Component-by-Component Evaluation
On-Farm Development Support – Technology
Advisory Services. The design of the sub-component sees
a substantial portion of the investment allocated to ISAR
(and an international research organization that works with
it): close two thirds of the total funding for the sub-component.
The majority of these funds are for staff salaries, vehicles,
motorcycles and equipment for the research station. While
this would not necessarily be a problem, with the M&E
system still not fully operational and its ability to assess
project impact limited, it is difficult to see what the impact
of these expenditures have been on the farmers and how ISAR’s
activities are linked to and respond to farmers’ priorities
for support. The other area for concern is that there is no
strategy or design of the On-Farm Development Support component
in the appraisal report and no detailed specification of the
implementation arrangements for the component, nor of this
sub-component. The only information contained in the appraisal
working papers (of the twin project) is a proposal for research
support. It has thus been difficult for the evaluation team
to assess whether the sub-component has performed well
or not. The project’s internal evaluation report indicates
that some 40 farmers were involved in technology tests but
that only six farmers adopted new technology (out of a planned
280) and that there were only 10 ‘innovator farmers’ (out
of a planned 24). In contrast, progress was achieved in participatory
diagnosis of production farmers’ problems (30 out of 47 sectors
planned) and a study on carrying capacity was completed. Nevertheless,
what is clear is that there needs to be considerable effort
made to develop a practical strategy for both this sub-component
and the component as a whole that demonstrates clearly how
farmers and other rural households participate in the programme
and benefit from its implementation. Working Paper 4
attempts to describe how such technical advisory services
could be implemented but the Programme management team will
need to carry this work forward into detailed monitorable
work programmes.
On-Farm Development Support – Agricultural
Production. This sub-component responds to one of the
most critical constraints for farmers in the province: poor
seed quality. It is rather support for improved seed production
and distribution than of the broad-based agricultural production
that the title implies. The design of this programme has been
a problem, as indicated by the minimal achievements attained.
While a reported 130 seed multipliers were identified, trained
and contracted, the amount of seed produced was only enough
for sale to the project with none being sold to the communities.
The programme also included provision for the construction
of four seed multiplication and fertilizer distribution centres;
however, none were constructed. Seven sites have been identified
for construction of such centres in the second phase, but
there are still outstanding questions about how these centres
will be managed and whether they will become public or private
assets.
A second programme included under this sub-component
is: seed and fertilizer distribution to vulnerable households.
This was planned to be a substantial programme with 12 000
households to receive a package of seeds (only 7 400
households received packs), seedlings and inputs, as a grant.
While these households are definitely among the poorer households
in the province, the logic of providing production inputs
free needs to be reassessed. The packages have not always
been used effectively and training in use of the package should
be a prerequisite in any future programme.
On-Farm Development Support – Veterinary
and Livestock Production. The limited budget and attention
to the design of this programme in the appraisal belie the
importance that livestock has in the province – it is the
major livestock production area of the country. A wide range
of activities were identified, but no detailed design was
done and the activities lack implementation strategies. Furthermore,
the sub-component was under-budgeted. The programme includes
the following activities: (i) delivery of veterinary services
by selected community members through the funding of a ‘community
animal health agents (‘paravets’)’ scheme; (ii) reinforcing
the capacity of the districts’ veterinary technicians to carry
out participative diagnostic of livestock’s owners constraints,
(iii) establishment of a stock of veterinary drugs and vaccines,
(iv) experiments and trials on the production and improvement
of fodder crops and improvement of rangelands, (v) promotion
of animal traction through demonstrations, (vi) support the
improvement of local breed through the introduction of quality
genitor of improved breed, (vii) construction of four
pilot livestock markets; and (viii) exploration of improved
livestock production technologies/practices. Relatively few
of these initiatives were implemented and those that were
started with a considerable delay. The paravet scheme, for
example, did not start till the third year but with the contracting
of an international NGO ‘Veterinary without Border’, the programme
was successful in developing a training manual and selecting
and training some 40 community animal health agents and providing
them with starter kits. The project also co-financed a vaccination
campaign against CBPP, trained of task force members and established
12 on-farm trials and demonstration of fodder crops. The formulation
of the appraisal report nor the working papers related to
livestock did not provide the implementation and phasing of
such activities. Considerably more attention and budget needs
to be allocated to livestock development in the second phase
with particular attention given to commercialisation, the
forming of livestock owners associations and the support for
the milk commodity chain.
On-Farm Development Support – Forestry.
The implementation of this sub-component was one of the more
successful and the performance of the international NGO charged
with its implementation was good. The objective was to respond
to the increasing demand for wood products while addressing
the problem of environmental degradation caused by the indiscriminate
cutting of forests for building materials for new settlers
and opening of new grazing and cropping lands. The target
was to establish 2400 ha of district forests and woodlots
and fruit tree plantations for vulnerable households. To achieve
this, it was planned to establish individual 40 nurseries;
due to a very attractive price of seedlings (twice the going
price), 164 nurseries were established that produced sufficient
seedlings to plant 1227 ha of forests and 32.5 ha of fruit
trees, reflecting an achievement rate of 102% and 81% respectively..
This good performance it provides valuable experience for
implementing a second phase forest management programme.
Rural Financial Services. The major
focus of the component during the first phase the formation
of Group Revolving Funds (GRFs) and women’s investment funds
(WIDs). The project was extremely successful in forming both
types of bodies with some 169 GRFs and 654 women groups formed
in five of the eight districts, which indicates the strong
need for such institutions. But, they operate primarily as
community savings associations, some based on the traditional
tontines with rotational withdrawal of savings by one
of the members and others taking in savings and providing
loans to the members. However, in both cases, the sums are
small and very often the uses of the funds are social rather
than economic. They provide an important facility for rural
households and are also one of the few initiatives that have
allowed women to develop their own savings and in some cases
get access to credit. Two issues arise: firstly, the new
Central Bank legislation would appear to make these institutions
illegal unless they can meet the registration requirements
(which for most is extremely unlikely) and secondly, the practice
of providing grants to the WIFs is now recognized as inappropriate
and goes counter to sound financial management and sustainability
and they need to be translated into credit. This initiative
formed part of the first sub-component for rural finance:
development of microfinance institutions. The implementation
of the rest of the component has been considerably delayed,
due to a number of constraints, including: (i) difficulties
in agreeing on an appropriate modus operandi for implementation
of rural finance between the project and IFAD, which delayed
the recruitment of microfinance institutions (MFIs) to provide
support to the project, including technical assistance to
GRFs and other solidarity groups; (ii) lack of staff
in the PCU (one single person in charge of capacity-building
and financial services components); and more recently, (iii) difficulties
in defining an implementation strategy compatible with the
Central Bank instruction on microfinance and with methodologies
currently applied in the microfinance sector in Rwanda. There
has been no provision of funds to the groups (as promised
as part of the project support) and no provision of training
in accounting and financial management. However, the project
has recently contracted two MFIs – Duterimbere and CSC Ugama
– on a trial basis for a few months, each responsible for
pilot programmes in two districts. They have yet to start
operations. The strong interest in savings and credit groups
and the possibility of developing new approaches to the provision
of financial service through the two MFIs should be used as
a basis to create a new financial services programme in the
second phase.
Project Management Support. This
investment component provides for the salaries of the PCU
plus the associated vehicles, equipment and some civil works
(offices and two houses). The original staffing proposed for
the PCU was dramatically underestimated, even for only four
districts, with only three professional staff provided for
(project coordinator, financial controller and project evaluation
officer). Even with the addition of the twin project the staffing
was only increased by two (deputy project coordinator and
community development officer). Project management is discussed
further in the following paragraphs.
|
