Interim Evaluation

The project area covers the Hoima District, which is located in the Uganda's equatorial zone. The altitude ranges from 670 m to 1 500 m. The total area of the district is about 10 000 square km. Of the 6 600 square km of cultivable land, only 14% were under cultivation in 1987. The climate presents a bimodal rainfall pattern with the possibility of two growing seasons during the calendar year. The average annual rainfall ranges from 800 to 1 500 mm depending on the year and the ecological zone. Zone II, the south and south-eastern sections of the district, enjoys a benign rainfall pattern; soils of gritty, sandy and gravely loams are of intermediate fertility with the highest potential occurring in the area to the east of Hoima Township. Zone III, the northern and western sections, has generally shallower clay loams of intermediate to low fertility and a generally lower rainfall.

Project design and objectives

Target group

The project area population in 1988 is estimated at about 379 200 people (61 000 families) at design. The project target group consists of the people who are most at risk from hunger, malnutrition, ill health and a general low quality of life: mainly young mothers and children. The project proposal to provide primary health centres facilities and services should cover the women and children of about 39 000 rural families out of total number of 55 000 who do not already have access to basic health care (the estimated balance of 16 000 families live in too remote district areas

The project's economic activities would be targeted on the women of an estimated 51 000 of the 70 000 families unable to pay graduated taxes of more than USh 4 300. The project training and credit would be provided for about 12 040 or 17% of these families (31% of project target group).

Objectives and components

The project aims to:

(i) reduce morbidity and mortality among the most disadvantaged people in Hoima and Kibaale Districts, particularly rural women and children;

(ii) improve their nutritional and health status;

(iii) increase both farm and off-farm income; and

(iv) establish the framework for integrated, district-based development of social services in Uganda.

The project has five components: (a) health, (b) water and sanitation, (c) agriculture, (d) community development, (d) infrastructure support, and (e) management support services.

Expected effects and assumptions

Welfare benefits include access to improved health facilities and health education. The closer proximity of potable water supplies would create the opportunity for these families to spend more time in productive work and to perform that work more efficiently. The health programme would generate incremental output approximated at USH 1 193 million annually.

Project implementation would improve cultivation practices on an estimated 18 600 ha farmed by 12 040 smallholders. Adoption of the improved practices of the MIP by the beneficiaries would result in an increase in the value of production of about USh 65 700 (38% on average family incomes in 1988) for the typical producer in the Banana zone; and USh 57 200 (35%) for the average Banana-Millet zone. At the projected rate of take-up, incremental production after deduction for seed retention and storage losses is expected to reach 1 670 tonnes of maize, 2 080 tonnes of plantain, 760 tonnes of beans and 790 tonnes of groundnuts.

Evaluation

An Interim Evaluation Mission of the Hoima District Integrated Development Project (HDICDP) visited Uganda from 3 August to 6 September 1996. The Interim Evaluation Mission specific terms of reference (TOR) were to (a) analyze project performance and achievements, particularly in relation to project beneficiaries; (b) assess the project impact on the development and social well-being of the communities in the project area; and (c) examine the sustainability of project activities in terms of cost- effectiveness, community participation, cost-sharing and recovery.

In addition to traditional methods of evaluation, the IEM applied Participatory Rural Appraisal (PRA) methodologies to carry out a Participatory Evaluation (PE). The PE was conducted in six sub-counties and 12 villages in the project area to ascertain the beneficiaries opinions and perceptions on project interventions, participation, and sustainability.

Implementation context

Uganda in general, and the project area in particular enjoyed peace and stability during the period of project implementation. In addition, some of the fruits of the structural adjustment programme, e.g. increased agricultural produce prices, have been felt by producers at the farm-gate. This has been assisted by the roads improvement programme executed by the project as well as the government and other donors such as Irish Aid.

The project had been noted for design flexibility during implementation. Adjustments in design included (a) the construction instead of rehabilitation of health units, (b) incorporation of health programmes for HIV/AID, malaria control and registration of births and deaths, (c) improved home programme, (d) the abandoning of the minimum input package, (e) the restructuring of the revolving credit fund, (f) the establishment of the liaison office in Kampala and (g) reducing the number of schools in the water sanitation programme.

Project achievements

Project implementation started only in 1992 and gained momentum in 1995, consequently little impacts are expected at this stage. It is also difficult to identify impacts attributable only to the project interventions, since there are other programmes funded by the government, as well as other donors and NGOs in the project area. Furthermore, the impacts brought about by the prevalence of peace and by macro-economic adjustments, particularly on development of markets and increases in agricultural produce, are more significant than those to which the project had contributed. Notwithstanding these factors, the Interim Evaluation Mission noted project impacts on various fronts.

(a)Health Indicators

Statistical Indicators. The only available data concern Hoima district where AVSI has started implementing the HMIS. The catchment areas of all the health units in both districts have been defined, but indicators cannot be calculated because there is no one to input the data from the peripheral health units. The main areas of improvement concern immunisable diseases (and mortalities) and morbidities related to water-borne/water-washed diseases.

Immunisable diseases. According to the FDP, only 6% of the children under five had been vaccinated against tuberculosis (BCG) in 1989. The coverage in 1994 was reportedly 107%, but only 70% were found to have verifiable scars. Although the target for BCG vaccination is 100%, 70% remains a huge improvement. Vaccinations against measles, polio (OPV) and diphtheria/ whooping cough/ tetanus (DPT) are verifiable only by consulting the child's card. The same discrepancies are noted but the verified levels for 1994 again reflect significant improvement and over 80% of the women had had at least two shots against tetanus. Anti-polio was not even given in 1989. Coverage in 1994 was higher than 85%. The project's contribution is through sensitisation on the benefits of immunisation.

Mortalities have certainly declined due to better immunisation but project activities have probably not had much impact on maternal mortalities except through vaccination against tetanus. The existing maternities are not adequately staffed and there is little hope that the project's seven units will fare any better. The training of TBAs can be expected to have a significant impact but their number is still small.

Morbidity reduction is the front where there seems to have been significant improvements that can be linked to the project. Apart from the immunisable diseases already mentioned above, both the HMIS-Hoima data and beneficiaries report the virtual disappearance of skin diseases, significant reductions in eye diseases, diarrhoea and malnutrition. Kwashiorkor is no longer considered a problem. The only negative trend concerns AIDS and the related opportunistic diseases (tuberculosis, measles, herpes zoster, etc.).

Access to safe water and sanitation. Less than 40% of the population of Hoima district has access to safe water and the situation is considerably worse in Kibaale district. School boreholes will not make much difference to this situation and the intended demonstration effect of the latrines is practically zero. Impact on child health will be nil for as long as they continue to return home to unsanitary and unsafe conditions.

(b)Agricultural Support

This component has clearly been least effective in carrying out its mandate and creating an impact on the target population. Adaptive trials and demonstrations have not been effectively undertaken, and therefore the demonstrative impacts on farmers is virtually nil while improved seed and plant materials have not been effectively propagated in sufficient quantities to have a measurable impact on local farms. The extension service has failed to reach the farmer. Therefore any increase in food production cannot be attributed to HDICDP.

(c)Access Roads

As anticipated by the FDP, the impact of access tracks on community services and marketing was favourable. Beneficiaries confirmed that CATs have bolstered local trade, with more traders coming in, produce buyers renting shops for storage purposes, agricultural produce prices increased significantly..

(d)Revolving Credit Fund

Disbursements of credit from project resources has not commenced, but as the evaluation analysis has shown that considerable positive impacts are felt from the formation of women's groups. In social terms, cohesive women's groups exercising regular savings and conducting regular meetings are established and trained by CDD and the Cooperative Department. These groups gained considerable experience in handling financial transactions, managing small-scale businesses and diversifying their portfolio of investments. The situation has matured for a full-scale distribution of credit.

(e)Coordination & Management

The overall project management has been satisfactory. The PCU has increasingly assumed an implementation role as opposed to the mere coordination originally assigned to it. The achievements of the PCU are a result of a remarkable continuity as well as smooth hand over and cooperation among the top management. But there remained a serious problem with the finance management. It was only after a lapse of four years that a satisfactory arrangement could be worked out and still pending the execution of the advisory services to the district treasury.

Effects assessment and sustainability

Beneficiaries' Perceptions. Participatory evaluations conducted with a view to capturing beneficiary perceptions of the evolution ofhealth over the past five years confirm reports of a better situation in Hoima, probably in the last two or three years. The ailments that beneficiaries felt had improved the most over the past five years are the same ones that are indicated by the HMIS-Hoima data. Measles incidence is ranked at 25% or less of the levels recalled for 1991, whooping cough, polio and tetanus at less than 15% for Kibaale and even less (even zero) for Hoima. Significant reductions are also indicated for kwashiorkor among children. The reductions in skin and eye diseases are significantly greater for Hoima than for Kibaale where water problems are more severe. Participants mentioned immunisation and sensitisation and training on hygiene (cleanliness), water-handling and nutrition as the main reasons for improved health, as well as availability of drugs and creams at the health unit and in the shops.

Effects on beneficiaries incomes. The trend analysis of economic status (incomes, declining hardships, markets for crops, etc.) showed that the year when things really started improving is often the one when a major road was opened. The improved economic environment was also attributed to the effects of community sensitisation/ mobilisation and women's group formation. The project was responsible for supporting roads improvements in many but not all of the communities through the provision of tools and culverts, and CDD has worked steadily with the women's groups for over two years.

Specific effects on women. Many of the trends show that groups already existed in the late 1980s, when CDD started this activity. Project support to CDD has been highly instrumental in reviving this approach towards the development of women.

Sustainability

Staff allowances and operational expenses. The project generously rewarded its own staff and, to a lesser extent, the staff of the local implementing agencies. There is a broad consensus that, while some incentives are necessary, the current level will not be maintained by the districts once the financial responsibilities of these activities are transferred to them. The situation would be further exacerbated with the termination of the technical assistance support and the discontinuation of the training programmes.

Whereas community sensitization and beneficiaries training under the CDD proceeded well, the financial aspects implied mixed results. Communities shared the costs of the construction of the health facilities, the community centres and the access tracks and crossings and provided communal labour on voluntary basis but failed to ensure the recovery of services costs.

The district level support is not likely to materialize because the capacity for revenue collection has increased only marginally. Only limited training was provided by the project. The revenue collection staff was retrenched at the operational level of the sub-parishes. Furthermore, the tax-base remained small, and its expansion was restricted by political interventions. It is logical to conclude that without huge influx of financial support from the centre, the Hoima and particularly Kibaale districts would find tremendous difficulties in sustaining project supported services in health, sanitation, water, education, agriculture and community skill development.

Main issues and recommendations

Main issues

Lack of funds. Over-expenditure of project coordination funds as compared to design constitutes a threat to both sustainability of project activities' management and the project's capacity to reach its original closing date (31.12.98).

Staffing. The staffing of the PCU has been adequate, but not so for districts and implementing agencies. The explanation lies with the incentive package, which has been more than adequate for PCU staff and deficient to the district and agencies staff. Whereas the project catered in full for the PCU staff, it could only provide limited support to the district bureaucracy, which are short of resources to maintain an adequate reward system.

Finance. Financial management is currently normalizing after years of difficulties, mainly due to unfortunate experiences with TAs. A major consequence of TA discontinuity has been the absence of TA provided to DA Treasury staff, which hinders sustainability in view of increased DA management responsibilities after project closure. The mission was informed that a Financial Advisor is being recruited to address this issue.

Procurement. The procurement of goods and services has been an important source of delays in project implementation. Surprisingly, at the beginning of project implementation, the procurement procedure had taken shorter time; but during peak of project implementation, considerable delays were experienced. Some components such as the health and sanitation, revenue collection, roads, drilling programme for water have been affected significantly in pace and cost of implementation.

Health. The effectiveness of the infrastructural improvements supported by the project are heavily conditioned by quality of staffing. Central government wants to pull out of district health matters and the districts do not have the financial resources to take over. There is evidence that very few of the existing health units will be able to finance themselves unless they raise fees to double or triple current levels.

Agriculture. The BDFI has not developed its demonstration/trials activities to a level that would enhance training. Without proper and convincing demonstrations, the benefits of residential training will be minimal and will not prove to be cost-effective for the project. Training was recommended to be brought down to the field level yet little systematic training has reached the group level. Research linkages have been weak and there is a notable lack of demand- driven research from the farmer level. Despite the support of the AEP to the district UES, there remains a poor link between researchers, extension workers and farmers.

Access Roads. Given the geographical distribution of CAT programme activities to date, it appears that the more accessible and better-off areas have been reached. Although this is understandable for various reasons, it does raise the issue of whether a greater effort can be placed on targeting of the most disadvantaged communities in Hoima and Kibaale.

Credit. Institutionally the structure of the RCF is still weak. The SIDAs - the association of the groups - do not have a recognized legal status, and hence the difficulty of a formal linkage to UWFCT arises. This is critical for loan recoveries and hence the sustainability of the credit scheme. Moreover, it becomes even more critical when the project winds up and there is no envisaged linking with either the DA or the banking system in the area.

Monitoring & Evaluation. (i) No evidence of support to the project by central M&E services was found by the mission. (ii) The baseline survey and the credit study are not satisfactory. (iii) Evaluation has not taken off yet. In this respect, the PCU has stressed the absence of TA support on M&E which was foreseen at the design stage but never materialized.

Recommendations

Coordination and Management (C&M). The project accommodated the split of its area into the domain of two districts at a cost. It would be more practical to limit future project intervention to one district, in which case C&M would be facilitated by the integration of the project activities into the normal functions of the district administration. However both districts qualify for the follow-up phase and hence the project activities should be undertaken by the district agencies directly, perhaps with some assistance from a procurement manager.

Regarding the optimal top-up of salaries and project allowances meant to motivate PCU, DA and IAs staff in implementing project-related activities, three principles should guide project leadership in this respect. First, project allowances should be based on staff performance. Secondly, they should be set at a level that the government can sustain with its own budget. In particular, project allowances should take into consideration declared government policies, on civil service "minimum living wages". Finally, they should be in line with other donors' allowances given to their own projects' staff.

As a matter of principle, the GOU should immediately terminate collection of income tax from donors' funds and UNOPS should stop honouring such payments. The total amount paid is approximated at above USD 100 000 (amounted to USD 94 000 for period 1992-95), should be reimbursed by the GOU to the project. This is crucial for the support of project activities, since the project is short of finances to continue to the design closing date of December 1998. In the future, such lapses should be avoided, by applying the spirit and principles of donor financing.

The project should concentrate its efforts on the in-service training for women's credit group, SIDAs, CAs and revenue collection. Since the DA will embark on the selection of district staff to run activities at all levels, the project should limit its training to enhance capacity building at district level. The on-going training of health and community development staff should be implemented as planned.

Finance. The design paper was overambitious in its assumption that the Financial TA had to (a) manage project financial services and (b) provide TA to the District Administration staff. The two functions should be separated.

Both at the project and at the district level, the financial officers should have the ability and the authority to design an allocation plan of project/district resources available every year. If budget limitations established at the design stage turn out to be unrealistic, the project M&EU and PCU, as well as IFAD and the Cooperating Institution should spot the issues in time for new budgetary plans to be based on proper analysis, then discussed and approved.

Procurement. There is a need to review Government procurement procedures with the objective of reducing the time and hassle. The GOU has already started a review process for CTB procedures. Indeed the CTB has improved on its procedures over the last five years by establishing technical committees at project level and practically endorsing the technical committee recommendations for the final award. The Government should take a step further, whereby the decision of the project procurement committee, in which a representative of the CTB participates, should be considered final.

Development of Community Participation. The CDD has performed effectively in the sensitization programme and has contributed directly in the implementation of the health, sanitation, water and women group formation for the credit programmes. It is essential that the project provides all necessary support to the CDD to continue its constructive role. The project support should include mobility, training, and allowances. Districts' authorities should redraw their priorities to ensure adequate number of staff, operational funds and prompt payment of staff salary and allowances.

The proposals for phase II made by project implementers are reasonable and founded on the experience acquired so far and many opportunities for low-cost approaches are identified. Of particular interest is the use of the community centres for agricultural training purposes.

Health, Sanitation and Water. The proposals for phase II presented by project implementers contain provisions for five new health units, only two of which seem to have sufficiently large catchment areas. The chances of self-sustainability of the units must be studied in detail in collaboration with the communities themselves before deciding on the design. The other proposals for the second phase are reasonable, though the impact of the envisaged training activities will be heavily conditioned by the quality of staff. Funding for additional health infrastructures should be subject to careful and fully participatory analysis of the community's financing capability.

Agriculture Support Services. Although the project has taken a step forward to address a demand-driven process, there is a lack of focus on the decision-making side of alternatives which address the production strengths and weaknesses of localized situations, together with the costs and revenues of alternative ventures. The BDFI Management and Technical Committee will have an important role to play in ushering in this change of attitude. BDFI must develop its adaptive trials and demonstrations to a much more serious level with better management. With a growing interest in livestock production systems in the area, BDFI should undertake more work in alternative fodder production and how it can be integrated into farming systems. A farming systems approach would demand a systematic PRA initial identification of farm management issues with participants at the start-up of the workshop for farm management. Modalities for decision-making on production options, with a focus on marketing, must also be developed.

The FEWs represent an existing structure at the sub-county levels (and lower) which remain under-utilized by the SIDAs. Since regular meeting of SIDAs is now established, SIDAs should be involved in the collection of data monitoring the level of contact of FEW's with individual groups. This independent system will help to ascertain the real level of contact between FEWs and groups which the project is supporting both logistically and financially through allowances.

Access Roads. The DOW has been dependant on the CDD sensitizing efforts of communities, the entry point for HDICDP components. The process could be enhanced by facilitating a PRA community mapping exercise. This would ensure broader consensus and views on the placement and prioritization of CATs, especially those of women who may be marginalized in the decision-making process. The importance of allowing the communities to work out a system themselves, rather than choose a blueprint approach for them should be emphasized.

Rural Credit. For the SIDAs to function as rural financial intermediaries, the issue of their legal status should be resolved immediately. Given the BOU concern with the institutionalization of the micro-financing institutions, the project should approach the BOU for assistance. One possible scenario is the registration of the SIDAs as credit unions who will be held responsible collectively and individually of loan performance. Registration as cooperatives should be an option of the last resort, because of their dubious performance in the past. As steps are being taken to register SIDA and groups, the PCU should agree with UWFCT to start a pilot credit scheme using the USD 120 000 at the project disposal.

The RCF should continue beyond the present contract of UWFCT (April 1998). If UWFCT establishes credibility in administering the RCF, its contract should be renewed, possibly under the Masindi Project, or under the auspices of the BOU, which is in search for a modus operandi for micro-financing. The contract for the second phase of UWFCT should include a stipulation for graduating and linking groups with the formal banks in the project area, if such services have been sufficiently developed.

Monitoring and Evaluation. It is important that M&E Units be established at central level (MOLG, MPED) and linkages reinforced with the M&E function at the project and district (DPU) levels. Training, TA and logistical support should be provided to DPUs to enable them to perform M&E functions, if the DA is to replace the PCU in a potential second phase. More efforts should go into the selection of performing consultancies, definition of TORs and quality control of the work delivered. Definition of M&E procedures, identification of impact indicators, logical framework should be ready by PY1.

The evaluation function of the M&E unit should be given due consideration through identification of appropriate indicators, collection of the optimal amount of data and systematic analysis of data. The evaluation task could be assigned to the central M&E units thus fostering operational linkages with the project and districts units. The selection of agencies or consulting firm to undertake base line or field surveys or evaluation studies should be based on merit and performance.

Lessons learned

It would be more appropriate and less problematic if the project is located within one district as it was intended originally for the HDICDP; but since project activities in both Hoima and Kibaale districts need to be consolidated and since both districts are eligible, target areas of the second phase would have to incorporate both. Yet the project activities should be integrated with the respective district administration.

There are certain requisites to ensure successful coordination and implementation of project activities. These include: (a) project components and details of requirements and project activities should be spelled out separately for each district at the design stage; (b) the district administration represented by the office of the CAO should be directly responsible for the coordination of project activities, while the respective district level implementing agencies undertake direct field responsibility; (c) the project procurement should be undertaken on a combined basis for both districts and the services of a special procurement officer/TA could be retained for this purpose; (d) the district should retain and train qualified and dedicated staff; this process should be assisted by the decentralization policy which devoluted this authority to the districts; and (e) the control of project finances and the financial training at district and IAs levels should be separate functions.

For the purposes of effective implementation monitoring and future evaluation, the next phase of the project should start with the conduct of an elaborate BLS by a competent consultancy firm as a precondition for loan disbursement. IFAD may consider financing this activity on a grant basis from the special preparation facility.

In the following phase, physical construction and/or rehabilitation of health centres should be restricted. Expansion in health centres should be considered only under the following conditions: (a) an established community based demand and strong commitment for participation, materially as well as managerially; (b) strong elements of sustainability of the health centre services as reflected in (i) a commitment for cost-sharing and recovery; (ii) a large catchment area that would reduce the unit cost of services provided to affordable levels; and (iii) the provision of quality services in the health centre to attract users; and (c) for the referral system to function, the health centres should be staffed with qualified, dedicated and motivated medical staff.

The sensitization programme conducted by the CDD should continue to be the core of the second phase. The positive impacts of the community sensitization have been felt on all project activities, even though only meager project resources were made available to CDD. The sensitization of community should be greatly assisted by the completion of the training physical infrastructures (BDFI, RTC, and community centres).

Human skill development in terms of various training programmes and participatory community institutions (FEWs, FERs, CHWs, VHCs, HUMCs) have been established during the first phase; and these should be an integral part of the design of the second phase.

The major criticism of the second phase proposals presented by the PCU is the lack of input from these institutions into the needs and hence the design of the second phase. As participatory rural appraisal (PRA) may assist in the design of project based not only on the knowledge of the community of the resources, but on the communities' demands, it is recommended that a PRA be undertaken prior to the design of the project.

In a direct link to community sensitization, the foundation for a successful credit programme has been laid out, almost towards the end of the first phase. Not only should the implementation of the credit component continue to the end of the current project, but it should continue into the second phase. However, the credit design should reinforce some important aspects, which the mission already noticed, such as: (a) the graduation of the loanees from the project lending programme to borrowing from groups' own resources and local banks, through enhancement of their credit worthiness. This would also imply more banking services to become active in the rural areas; (b) sustainability of the credit operations should be ensured beyond the use of small funds from their own savings. This is achievable through reasonable returns to the loaned funds and prompt repayment of loans; and (c) the encouragement of diversification of businesses within and without farming, particularly in trading, marketing and small-industries.

The agricultural support programme has continued to lag behind in the current phase of the project, the essential elements for the programme to become effective are: (a) during the second phase, the agricultural support program should be placed under the direct control of the districts administrations; (b) the extension system using FEWs, FERs, CAs with BDFI providing the technical backup support should be activated; (c) improvement of the management of the BDFI, and the empowerment of the management committee are critical to the future role in the provision of technical solutions to the farmers field problems; (d) the BDFI should be required to develop an adaptive research programme that is farm system oriented, responsive to farmers needs and based on problem-solving approach i.e demand driven; (e) technical extension messages and the means to deliver them should be made available for extension workers to be effective; and (f) it is not necessary that the MIPs are provided through project financial resources. However, women farmers in need of credit for to obtain farm inputs could be assisted within sound financing arrangements.

Community access tracks and swamp crossings represent a genuine entry point for community organization. The current phase of the project had attempted social organization of women, the success of which is in large part attributed to the total community sensitization programme including sensitization of men to allow their women to participate and for them to take part in activities such as access roads and crossings or village water supply schemes. In the second phase, a broad village-based social organization should be established along with the women groups using these project activities as entry points.