Evaluation of Project Performance and Impact
Project Coordination and Institutional
Aspects
PCU. The performance of the PCU
in guiding/facilitating project implementation and providing
project oversight has been less than ideal. This is due to
a number of factors:
· Inadequate staffing and often insufficiently
experienced staff. As noted, the original budget for
staffing the PCU was totally inadequate. While this was recognized
early in project implementation and action was taken to strengthen
the unit, it was done in a piecemeal fashion and to-date the
PCU is still lacking in capacity to effectively manage the
project. The frequent changes in project coordinators did
not help – three in as many years – and in some cases the
shortage of sufficiently skilled and experienced staff in
the PCU has also handicapped its operations. While the numbers
have been progressively increased – the PCU now numbers eight
– there are still problems in achieving effective and timely
management of resources in spite of a high level of commitment
and hard work of the current staff.
· Time-consuming processing. The
time required to handle even very simple administrative and
financial matters can be excessive. This is recognized by
all parties, including the staff of the PCU. While staff shortages
and experience have been contributing factors, there are other
problems that need to be addressed: contract management procedures
and processes are slow, bureaucratic; and review/approval/no
objection processes (by the Tender Board and UNOPS) are lengthy
as sometimes is the submission of AWPBs by project partners.
Oversight/Coordination Structures.
The design of the project’s two-tier oversight structure involving
national and provincial steering committees needs to
be re-examined. It was observed in the review of the project’s
performance over the first phase – as presented in the internal
evaluation report – that the provincial steering committee
was too large to be a fully effective mechanism to support
the PCU and that a smaller committee or working group should
be considered. A question was also raised as to whether both
national and provincial steering committees were needed for
a project whose operations were fully contained within one
province.
Implementation Arrangements. The
basic arrangements for project implementation, as presented
in the appraisal reports, are effective and have allowed the
project to execute its range of investment programmes. However,
two aspects need specific comment: (i) the project’s
contractual arrangements with NGOs and (ii) the role
of districts in the project’s development process. As mentioned
previously, the performance of NGOs has been negatively affected
by the current management arrangements. The proposed role
of the I-NGOs as both funding and contracted parties for the
project has not been ideal and has caused tensions among the
partners. Further, the expected transfer of knowledge and
experience from I-NGOs to N-NGOs has been less than
ideal and the I-NGO consortium while providing a useful
forum for exchange of information among the members has not
brought the advantages originally foreseen. Working relations
have not been helped by the fact that the international NGOs
have been operating without contracts and from their own resources
for most of 2003. While there are a number of reasons for
this – including delays in and questions arising from performance
evaluations – and questions as to what extent the I-NGOs have
actually had to use their own resources, there has been nevertheless
a difficult working relation between many of the I-NGOs and
the project.
To implement the various project activities
has required that the project develop close working relations
with the districts – something that it has done diligently
and well. The districts have been partners in the planning
of project activities and in their implementation. But, to-date
it has been the project that has been making the decisions
and the districts that have been responding. The project identifies
the needs (together with the districts), identifies suitable
service providers, draws up contracts with the ones that successfully
respond to the project’s request for proposals and oversees
the performance of the contracts. The district is the recipient
and the project the deliverer. The nation’s decentralization
policy sees the district becoming the ‘decision maker’ and
consequently being accountable. To achieve this will require
a major change in ‘mind set’ in the project and a consequent
reorientation of project management procedures and resources.
Financial Management
Disbursements and Use of Funds.
Firstly, to-date (end-October 2003) only 13.6% of project
funds for Phase I have been disbursed with the highest
disbursement rate being for the following expenditure categories:
salaries (87%), operating costs (68%) and vehicles and equipment
(68%). This contrasts with the minimal disbursement in civil
work (2%), financial services (0%) and revolving funds (0%).
In terms of use of funds by component, the current percentages
of funds disbursed are: community development (12%), infrastructure
(10%), on-farm investment (18%), rural financial services
(0%) and project coordination (34%).
Annual Work Programme and Budgets (AWPBs).
Review of the process of preparing/approving AWPBs indicates
that while the project has been preparing and using AWPBs,
the draft budgets (for government and IFAD/UNOPS approval)
have been submitted several months late (contravening Section
3.03 of the Loan Agreement).
Contract Management. As noted above,
contract management is an area in which the PCU requires considerable
strengthening. It has been observed that: (i) there
is a difference in understanding regarding the detailing of
tasks in the NGO contracts, with the project claiming they
are sufficiently detailed and the NGOs claiming they are not,
which does not facilitate the annual task of performance evaluation;
(ii) a one year ‘Protocole d’Accord Cadre’ (framework
agreement) was signed with the consortium of I-NGOs in April
2001 that, in understanding of the I-NGOs, was in principle
renewable annually, however, it has never been formally renewed;
(iii) annual contracts with each service provider vary
in terms and conditions (e.g. advances range from 30% to 70%
with no apparent reason) and in the degree to which they specific
obligations of the contractee and its reporting commitments;
and (iv) the contracts entered into with public institutions
such as ISAR and the Department of Agriculture, Livestock
and Forestry (DALF) have no legal basis (as they are between
a government project and government agencies. The most important
observation is that considerably more effort must be put into
preparing contracts to ensure that the service providers understand
their tasks and obligations and that the PCU (and progressively
the districts) are able to effectively monitor their performance.
This requires much closer working relationships between the
financial controller and the technical staff of the PCU to
ensure that the technical requirements are well specified
in the body of the request for proposals and subsequently
in the contracts that are awarded.
Procurement. The analysis of the
procurement process has been constrained by an inadequate
documentation and filing system in the PCU, however, it is
evident that the time required to procure services has been
long: on average seven months. The delays are due to slow
processing in all three major steps in the procurement system:
request and evaluation of bids, approval by the National Tender
Board and issuance of ‘no-objection’ by UNOPS.
Accounting System and Internal Control.
The general organization of the accounting is satisfactory
although the PCU has not as yet fully converted to the TOMPRO
software. The internal control system currently used by the
PCU while effective is too lengthy and should be simplified
without altering the quality of the control. It has also been
observed that the project is too centralized in its organization
and operations; there is considerable scope for a responsible
delegation of power within the PCU to facilitate processing
efficiency. Plus the processing of performance based contracts
and their evaluation was too heavy to be implemented properly
by a small PCU.
In summary, it can be stated that the PCU
has struggled with the financial management of the project.
Due to overly centralized decision making and cumbersome and
lengthy processing procedures compounded by lack of relevant
project administration experience of some key staff and understaffing
problems, there have been accumulating delays. This has had
serious repercussions on the project’s delivery of services
and investment, as is evidenced by the low disbursement rate
up to October 2003.
|
