The evaluation was mounted by OE at the request of the Belgian Survival Fund with a two-fold objective: to assess and document Project impact and sustainability; and to develop insights and recommendations that will serve to improve Project activities during the remaining implementation period, the design of future projects and programmes in Tanzania. It is one of a series of evaluations based on the new IFAD Impact Evaluation Methodology. This framework: primarily stresses rural poverty impact, as a main measure of project success; includes assessment of performance of the Project, encompassing key criteria of relevance of objectives, effectiveness of achievement of objectives, and efficiency in implementation; and assesses performance of partners, that is, of IFAD, its cooperating institution, government, co-financiers, project management and other participants. Preparatory work by OE in a desk review of documents culminated in an Approach Paper that identified the main issues and elements of the evaluation. Subsequent briefings stressed that evaluation was to be independent and objective, emphasising actual operations and real outcomes on farms and in households and communities - and not just on a stereotyped analysis of records, reports and performance figures.

The evaluation took place in November/December 2002 and involved wide ranging and detailed discussions with key informants across the spectrum of participating parties; interviews with more than 200 farm families; over 30 in-depth group discussions; and completion of the new impact evaluation matrix forms in all five districts and, in brief format, in a village. Preliminary Mission findings, issues and recommendations were discussed at a Regional Wrap-up Meeting and Stakeholder Workshop in Bukoba; and, as amended, at a National Wrap-up Meeting. The findings presented here reflect the interactions with: 60 district, farmer, and Project delegates; and various senior ministry and partner agency officers in Tanzania.

Development and project perspective

The political system and economy of Tanzania have been transformed since the late 1980s and the country is acknowledged among the leaders in sub-Saharan Africa in economic reform and social equity. Since 1996, growth in real GDP has averaged over 4%, but GDP per head remains at around USD 250 and basic needs poverty still affects some 36% of the population. The predominantly subsistence-based agriculture sector has grown at 3% per year, employs 80% of the workforce and accounts for around 50% of both GDP and all exports. The planning of KAEMP coincided with decentralisation under the Local Government Reform Programme. In 2001/2002, the development policy framework was further enhanced by adoption of a Poverty Reduction Strategy Paper (PRSP); the Agricultural Sector Development Strategy (ASDS); the Rural Development Strategy (RDS); and the new Agricultural Sector Development Programme (ASDP).

All of these emphasise devolution, good governance, district self-reliance, private sector/NGO participation and gender equity in poverty alleviation. The clear PRSP priorities for development investment are: education; health; water; agriculture; roads; and the legal system. It is implicit that agriculture, as the main productive and income generating rural activity, must play the pivotal role in poverty reduction. Districts formulate their development plans on the basis of these reforms and priorities. Tanzania as a country and Kagera as a region are unusual in Third World development in enjoying a plentiful supply of land of reasonable quality, generally favourable climate and a varied, but benign natural resource base. This includes strong customary rights of use and effective security of tenure for farmers, although land is nominally state owned.

Main design features and changes

Project rationale and strategy. The major determinant of the rationale was the need to combat the adverse effects of the refugee influx of about 600 000 people, representing a nearly 40% increase of the local population. Their presence was exacerbating: the already declining soil fertility and agricultural productivity; depletion of vegetative cover; scarcity of safe, clean drinking water; the pressure on local health services; and the deteriorating condition of rural and village access roads. The severity of refugee impact was such that a strategy of direct action was needed to assure food security, household incomes, reversal of degradation and district ability to provide services.

This implied: in agriculture, supply of improved seeds, planting materials and inputs; and introduction of simple new husbandry techniques, such as integrated pest management (IPM) and integrated plant nutrition (IPN); in environment, strengthening of forestry staff, seedling production and tree planting; and in infrastructure and services interventions, tackling the constraints to productivity, marketing and well-being of: lack of equipment, supplies and training for health facilities; the time and effort involved in safe water collection; and the poor condition of rural roads. Implementation was lodged in the devolved local government system, as a means of district capacity building and orienting staff to participatory processes.

Project area and target group. Kagera is the remote north western region bordering Uganda, Rwanda and Burundi. Road access is difficult and services scarce in the rural areas. The Project area was defined as all five districts for the agricultural, environmental and project facilitation activities; but only four districts for health and three for roads works and water supply. The region has a favourable climate and land for crop production. The important agro-ecological zones are the Highland, High/Medium Rainfall Perennial Banana/Coffee Production zone; and the Lowland, Medium/Low Rainfall Annual Crop Production zone. At appraisal, the indigenous population was 1.6 million people in 290 000 households, dispersed in 550 villages with densities varying from 25 to 100 people per km2. Smallholder subsistence farming dominated the economy.

Poverty and welfare data indicated that the region was among the poorest in Tanzania. Over 40% of households were frequently deficient in staple food supply and over 80% were classified as poor. The target group universe was defined as between 170 000 and 195 000 households that comprised: the 30 to 40% of food-deficient families, with less than 1.2 acres of perennial crop or 5 acres of annual crop land - the primary target group; and the 40 to 50% of families that had some food security but no marketable surplus, with a perennial crop area of up to 2.5 acres or annual cropping of 5 to 7.5 acres - the secondary target group. Major causes of poverty were: distance from the road; lack of farm inputs, credit and extension; low prices, especially for coffee; inadequate access to safe water; scarcity of fuel wood; and lack of transport. The poorest families had: mud and thatch houses; limited cash crop production; no livestock; and pronounced child malnourishment.

Goal, Objectives and Components. The goal of the Project was to rehabilitate the region following the refugee encroachment and reverse the long term decline of agricultural production. The supporting objectives were to: improve household food production and incomes in a sustainable manner; support the process of improving environmental management through tree planting and water hyacinth control; improve access to safe drinking water supplies, health facilities and road access to villages; and strengthen the implementation and service delivery capacity of the relevant District Departments and assist development of a participatory approach to resource management.

The six year Project had four components and a number of key activities, as follows:

• Agricultural Development: (Base cost USD 7.8 million, 40% of total)

• Multiplication and Distribution of Seeds and Planting Material;
• Production Inputs and Equipment Supply; and
• Improved Crop Management - diffusion of IPM/IPN techniques.

• Environmental Management: (Base cost USD 1.6m, 8% of total)

• Communal and Individual Afforestation;
• Control of Water Pollution; and
• Institutional Strengthening

• Rural Infrastructure - Health, Water Supply, Roads: (Base cost USD 8.1m, 42% of total; plus USD 4.1m loan from OPEC for more major roads)

• Project Facilitation, Monitoring and Evaluation: (Base cost USD 1.8m, 9% of total)

• PFMU - equipment, salarial and operating costs of co-ordination and administration; and
• Districts - salaries, allowances, equipment, vehicles and training for district staff.

Total Project costs, including contingencies, were USD 20m, with the breakdown of financing as follows: IFAD loan - USD 14.8m; BSF grant - USD 2.5m; government - USD 2.1m; and beneficiaries - USD 0.6m. The OPEC loan of USD 4.1m for roads improvements was additional.

Implementation arrangements. The principal national level responsibility falls under the Presidents Office Regional Administration and Local Government Department, (PO-RALG), which, through the Prime Ministers Office, handles policy guidance and liaison with Ministries. At the region, accountability lies with the Regional Administrative Secretary (RAS), who is backed up in practice by: the Project Steering Committee (PSC); and the Project Facilitation and Monitoring Unit (PFMU). In fact, the latter exercises the major tasks of guidance, arrangement of technical support to participating districts and downstream agencies; and dialogue with donors. At district level, Councils and Administrations are prime movers in planning and implementation of activities, backed up by the small District Project Facilitation Units (DPFUs). Key players are the District Executive Director (DED), the Chairperson of the District Council and the District Administrative Secretary (DAS). The ward is the link between villages and districts, particularly for planning, but in some cases has not been fully involved in Project operations. There is a wide range of competence and understanding among district staff and within Ward Executive Offices and Development Committees and in Village Assemblies and Governments. Village Finance, Economic Affairs and Planning Committees are, in theory, the source of Project proposals, but have considerable problems of capacity and capability.

Policy, Institutional and Design Changes. The ongoing evolution of agricultural and rural development policy has not conflicted with the established philosophy and operating modalities of KAEMP. The main institutional changes have been the relocation of the Regional Administration and Local Government Department; and devolution of agricultural, water and other technical functions to districts; these have also worked in favour of Project progress. Support of decentralisation and participatory village and district planning are key elements of the RDS that have received strong Project backing, but as yet with only limited results. Project facilitation and management has been flexible and responsive, leading to a number of design changes; the major of these are: the appraisal logical framework has been replaced by a more useful version and the logframe approach has begun to be introduced in village and district development planning; the increase in traditional staffing proposed at appraisal has been replaced by the use of consultants and appointment of unpaid, but well trained and assisted Farmer Cadre extensionists (FCs); the original contract arrangements with substantial incentives and subsidies to local groups and individuals have been switched to a system of advisory support for SGAs, other groups and informal farmer/seed growers; and the original rapid multiplication programme for cmd resistant cassava cultivars has had to be suspended and a new mother garden and farmer multiplier method employed.

Further changes have been: establishment of a new input supply and credit scheme in response to the failure of the original sub-component due to default/non-repayment; amended pattern of uptake for water supply interventions; promotion of diversification of crops and enterprises in response to adverse market conditions for coffee and in some areas, banana; the recent start to address the issues of marketing, which were not fully appreciated at appraisal; and the assimilation of IFAD-sponsored initiatives in gender mainstreaming training and HIV/AIDS mitigation.

Main implementation results

Financial aspects of performance. Of the total funds available of USD 24.12 million, expenditures to date are estimated at USD 18.71m, about 78%. Expenditures by financier are in all cases over 75% of original budget amounts. Main divergences by category of expenditure have been: input suppliers credit of only 7% of planned; and excessive salarial and operating costs, at 122% of budget. By component, significant divergences are: lower costs for agricultural development, at only 43% of planned, due to the demise of input credit; water supply expenditure of only 49% of planned; and excessive Project facilitation and monitoring costs, which are already 104% of those budgeted.

The aggregate status of outstanding loan and grant funds available for disbursement from the main sources is estimated as: IFAD loan - USD 1.66m, principally allocated to civil works (USD 0.50m) and supplier credit (USD 0.60m); BSF grant - USD 173 000; government USD 340 000; beneficiaries USD 50 000; and OPEC loan - USD 600 000, making an overall total of USD 2.82m.

Summary physical achievements. In overall coverage, the Project is estimated to have had a significant presence in 300 of the original 540 villages of the region, that is 159% of the initial target; and to have impacted 190 000 households, or 1 million people, representing 111% of target.

Agricultural development. In Agricultural Development, uptake of improved seeds and planting materials has exceeded target except for clonal coffee and cassava. The targeted 24 Seed Growers Associations (SGAs) have been formed - and 18 of them so far registered. In improved crop management, the Project has fulfilled its targets in establishing 200 IPM/IPN farmer groups, of which 190 are still active, including several womens groups and IFAD/FAO-assisted farmer field schools, with total membership in excess of 5 000 at present, over 30% of them women. IPM/IPN techniques have been adopted by an estimated 6 000 farmers in addition to group members. The production inputs and equipment supply activity has proved unviable - only 160 farmer loans from 5 stockists were made, default was rife, and legal action is in train to recover outstanding loans and costs. Agricultural training and capacity building included provision of seed laboratory and soil testing equipment and facilities and refurbishment of the Farmers Training Centre at ARDI-Maruku. Two hundred and nineteen Village Extension Officers (VEOs) and other district staff and 9 000 farmers and one hundred and forty four SGA leaders have received training in Project activities and management skills. Of singular importance has been the appointment and training of Farmer Cadre extensionists; over 80 are posted, each typically with 30 to 40 farmer clients who add strongly to the extension impetus inculcated by the KAEMP group structure.

Environmental management. In Environmental Management, there has been marked success in communal and individual afforestation, with a total of 3 280 acres and 2.1 million trees planted, an over 120% achievement. The claimed survival rate is 78%; Mission estimates put early growth rates at 50% to 80%. Eight hundred and forty tree nurseries have been established, about half operated by individual growers and half by groups, schools and other institutions; at least 50 nurseries are now operating commercially. Uptake of wood saving stoves is estimated at 1 800 households or 52% of target; and water hyacinth control has entailed release of more than 30 million weevils following 14 pre and post-release surveys as against 18 planned. The on-target technical training of 26 forest department and extension staff; nearly 3 000 farmers in nursery practice; and over 2 500 village leaders on land management and soil conservation has been completed. The main shortfalls in the targeted objectives have been in district institutional strengthening, where staff numbers have not increased; and in communal resource management initiatives, where few have yet been attempted.

Rural infrastructure. Under Rural Infrastructure, interventions in health appear to have directly covered about 900 000 people, with a bias to women and children. In equipment supply, a near-target 87 dispensary kits and 7 health centre kits were provided, but with some duplication and unsuitability. In malaria control, 62% of the targeted number of bed nets were procured of which 60% has either been sold or issued free; all planned training was carried out in year one, 1998, with 100% coverage as planned of District Health Management Team members who were trained as trainers, but only 27 to 32% of the planned lower echelon staff - and none of the important traditional birth attendants - trained. The prescribed cost sharing for health services has not been widely accepted or applied.

Water supply. The water supply sub-component changed with the trend of demand, but discrepancies remain between achievements recorded and those able to be validated by the Mission. Technical design of works was to be done by district staff and construction by contractor/NGO. The competence of some technical design was questioned by the Mission; and much construction work seems to have been undertaken by district staff rather than by contractor, the reasons, costs and related conditions for which are unclear. The planned training of 18 technicians and 2 hydrologists became, in practice, for 46 technicians and one hydrologist, an achievement of over 230% - but a questionable one in relation to actual numbers of district water department staff employed. The numbers and types of schemes identified by the Mission as completed are: rehabilitated gravity schemes - 3, or 23% of target; new gravity schemes - 1, or 14%; deep wells - 17, or 95%; shallow wells - 51, or 230%; rainwater tanks - 67, or 223%; and protected springs - 72, or 205%. In addition, 3 pump schemes and 2 charcos dams were constructed, although such works were excluded at appraisal; and the Project is to provide equipment and supplies for three district water analysis laboratories not included at appraisal - and justification for which is questionable. The training and orientation of Village Water Users Group (VWUG) members covered 1 050 people. However, findings show that cost contributions prior to construction and cost sharing for operation and maintenance were - and are - limited; and administrative and management capability of groups and committees is rudimentary.

Implementation of the rural roads sub-component involved: participatory selection of sites; survey, design and supervision by consultants; and construction by contract. However, regional roads agencies played a major role after site identification with strong adherence to design parameters and commercial tendering/contracting processes. The OPEC loan funded rehabilitation of regional roads; the target total of 124 km was completed satisfactorily in 2001 by 4 contractors under 5 contracts. Village access roads works included minor items such as culverts and bridges and were accomplished by a mixture of mechanised and labour based technologies. Works cover 346 km and are about 95% achieved, pending remedial/completion inputs on one 13 km section. Standard of work has been satisfactory and the overall achievement of roads works has been commendable, although there are some reservations about drainage and gravel surfacing and out-sourcing of minor works. The sub-component included training in roads maintenance and repair of 4 roads engineers; 456 Village Roads Committee (VRC) members; and 760 village youths. All training has been done, but further training of roads technicians is required; and effectiveness of training/orientation of VRCs and groups responsible for repair and maintenance is yet to be proven.

Project facilitation, monitoring and evaluation. Principally comprises overall co-ordination, accounting and reporting of activities as well as provision of vehicles, equipment, salarial and operating costs for regional and selected district staff. Related procurement, contract tendering and funds provision exercises were fulfilled to target; staff appointment was completed in 1997 and the majority of procurement was done in 1998. Progress on district capacity building, beyond the sectoral training and project experience discussed above, has been less than expected. The M&E system has been changing continuously, with only rudiments of physical recording and management information provision maintained. District Monitoring and Evaluation Officers (DMEOs) were appointed only in 2001; training took place only in 2002; quarterly reporting against AWPB is currently not in use; and surveys, studies and workshops have not been executed as planned or been unsatisfactory in coverage and content. Refinement of M&E goes on, but the quandary persists of: an over-ambitious and complex system; limited competence and resources at districts; and absence of real participatory input. Training in participatory rural appraisal (PRA) for 65 staff took place earlier and PRA studies have been completed in 44 villages. Training in use of the logical framework for 20 staff was done in 2001. Since then its concept and practical use for planning, implementation, monitoring and evaluation have been demonstrated and applied within the districts and in 7 villages. The approach is promising, but community understanding and capability to apply it requires more assistance and long term support. Gender issues were given limited attention in Project design, but KAEMP has made some progress in enhancing womens participation. Further gender mainstreaming has been instigated by an IFAD grant in February 2002, under the Gender Strengthening Programme for Eastern and Southern Africa. Progress has been: training in gender awareness and approaches of 70 Project staff, including 19 women; appointment of 3 gender focal points per district; training of 200 village leaders and 22 women's group, SGA and roads committee members; and start-up of a dissemination programme for gender related messages to communities.

Rural poverty impact

Overview. In the absence of comprehensive M&E records of Project impact, the draft Household Budget Survey of 2001 provides socio-economic data for Kagera that show that from 1991/92 to 2000/01, the share of population below the food poverty line declined from 23% to 18%; and that below the overall basic needs poverty line, from 42% to 29%; the latter figure compares with a national level of 36%. The median annual rural household income is put at TZS 801 000 (USD 809) and the mean annual consumption expenditure at TZS 756 000 (USD 763), indicating the precarious balance between family resources and needs. Data also indicate that: income inequalities are worsening; only 13% of houses have modern walls, 2% mains electricity and 31% improved water sources; life expectancy remains at 45 years; maternal mortality at 228 per hundred thousand; and infant mortality at 127 per thousand, 33% higher than the national average.

Impact on physical and financial assets. The impact has been positive, mainly because of : increased farm productivity leading to food self-sufficiency, availability of surplus crops for sale and higher household incomes and ability to invest; establishment of tree nurseries as a source of income and tree planting as an income generating and asset accumulation venture, related also to occupation of additional village land; and roads improvement that has greatly facilitated market access and increased farmer/trader contacts and petty trading. These gains are based on: introduction of sound, farmer friendly, no-cost or low cost technologies; innovative extension methods; widespread sensitization and training of farmers and communities on the conservation and economic potential of tree planting; and support of productive processes by improved road access.

The evaluation recognises two categories of outcome: first hand, direct beneficiaries and benefits; and secondary, indirect beneficiaries and benefits, part of the trickle-down effect. From agricultural activities the former are estimated at 6 500 households; the latter at an additional 15 000 households. The first hand and secondary beneficiaries from environmental actions are estimated at 2 400 and 20 000 households. Roads beneficiaries, whose benefits are more tenuous, are estimated at 35 000 and 100 000 respectively. The quantum of physical and financial asset benefits and wider farm and household impact have been estimated from models set up to illustrate the effects of the principal farming and environmental interventions. The results are tabulated below.

 

 

2.20

Model Category	Without Project USD	With Project USD	Incremental %
Net Margin	Return to Family Labour Day	Net Margin	Return to Family Labour Day	Net Margin	Return toFamily Labour
Upland Perennial Crops System/Farm	334	0.76	592	1.05	77	37
Lowland Annual Crops System/Farm	420	0.70	609	0.88	45	25
Tree Nursery	na	220		Na
Tree Plantation	na	Peak Investment	Break-even Year	Financial IRR %	Na
731	12	18

The positive impact on physical and financial assets is shown in: ownership or rights of use of additional land, tree nurseries and tree plantations; livestock, especially cattle, purchase; better housing, ranging from steel sheet roofs to brick built houses; reduced indebtedness; ability to pay school fees and buy better clothes; and ownership of furniture, utensils, tools, bicycles, and radios. Project contribution to increased physical and financial assets is substantial, 3 on a scale of 1 to 4.

Impact on human assets. There have been some positive effects on human assets: agricultural and environmental activities have contributed to both food supply and family income, two key determinants of nutrition and affordability of health and education; but for health and water supply, the performance has been inadequate. Activities in health were expected to result in reduced incidence and severity of malaria, peri-natal complications and mortality, other common diseases and improved child nutrition; in the event, the evidence of significant impact is not yet conclusive. Nor is there evidence of decline in water-borne diseases, despite the intervention in water supply, which should have reduced the collection burden for women and children and assured safe and clean supplies. The Mission estimates that only 61 000 people, or 10 000 households -- as compared to the 98 000 beneficiaries estimated by the PFMU -- are covered by improved schemes and since many of these did not change accessibility, and most have been found to be contaminated, work burden alleviation and clean water availability have been limited. The Project has provided financial means that add to the rise in primary education brought about by government policy; and engendered probably the major change in human assets in the uplift in knowledge, skills and confidence of participants. The evidence is seen in: the competence and discipline of take-up of interventions; the level of participation and interest in new developments and social change; the success of the Farmer Cadre system; the ability of men and women farmers to present themselves and their views; and the coherence and resilience of the group structure. A similar, but as yet small change in understanding and attitudes is being wrought among local councils, development committees and staff of formal district, ward and village institutions. The contribution to increase in human assets for health benefits is modest, 2; and that for water supply effects is limited, 1; for the imparting of skills, knowledge and self-confidence the contribution is substantial, 3.

Impact on social capital and empowerment. The Project has had a meaningful impact, emanating from the demand driven, participatory and group approach adopted in later years and from uplift of financial and human assets. All components have contributed, the main thrusts coming from: the IPM/IPN groups and SGAs; to a lesser extent, the environmental interventions with village groups and schools; and the health, water supply and roads O&M committees. The main weakness in social capital development and empowerment aspects is the limited degree of impact, so far, in the advancement of gender equity. There was also much more scope for Project activity on produce marketing problems, for which KAEMP was not equipped, or alerted to the need, to find solutions. Impact on social capital and people empowerment is assessed, in the round, as substantial, 3.

Impact on food security. Application of Project techniques has lifted crop yields and outputs significantly and typically moved poor, small farm beneficiary families from a position of seasonal food insecurity, to one of food self-sufficiency in most years. Estimated yield increases are over 100% for banana, 50% to 60% for beans; 30 to 70% for maize; and 40% to 60% for cassava. The farm models indicate that these yield increments lead, for the Upland, Perennial Crops system, to a substantial surplus of banana and grains/pulses available for sale; and for the Lowland, Annual Crops system, change a serious food deficit, to a marginal surplus in aggregate, with some grains/pulses available for sale. Impact on food security is substantial, 3.

Other impacts. The impact on the environment and common resource base where the Project has been active has on balance been positive. However, the severity of the original state of degradation and continuing profligate cultivation of erosion-prone lands and periodic burning of the bush leave much still to be done. The Project has had limited impact in community-based natural resource management, or in wider application of soil conservation measures and agro-forestry. The effort in water hyacinth control has reportedly been effective, diminution of infestation having been assessed at over 70% in the lake. Environmental impact overall is modest, 2. The impact on institutions, policies and regulatory framework has come from working through the organs of district governance, as well as with NGOs and the private sector. The Project has instilled much orientation, training and experience, as seen in increased effectiveness of extension and the start of use of the PRA and logical framework in local planning. There has been little change in rural financial institutions, but the formation of groups and SGAs has been a positive development. KAEMP has not played a major part in the evolution of new policies and regulations but has utilised approaches and techniques that could influence policy and regulation formulation in future. Impact on institutions and services is classified as modest, 2; impact on policy/regulations was not assessed.

Key innovative aspects were: in farming, use of low cost, farmer/environment-friendly techniques - such as botanical extracts as pesticides and farmer grown seeds; in extension, strong reliance on groups, encouragement of farm record keeping and business management, and instigation of the Farmer Cadre system; in environment, use of schools and commercial nursery and planting practices; in implementation, incorporation of Project activities in district development plans and promotion of participation and the logical framework approach for planning. All of these promise to be successful and highly replicable approaches. The positive signs of likely sustainability are: best practice applied with simple but high impact IPM/IPN technologies, conservation approaches and extension systems; development of sound group structures; the orientation and training given to staff, farmers, groups and local agencies; and the initiation of better systems of planning and management of district affairs. Potentially negative factors are: the questionable capability of water users and roads committees to perform operation, maintenance and repair; lack of acceptance of cost sharing; possible inability of districts to provide the funds to maintain development impetus; and the imponderables of adverse climatic and marketing conditions. On balance, sustainability is assessed as modest, 2. The ratings for innovation and replicability are substantial, 3, for agriculture, environment and project facilitation; but at best, modest or 2, for infrastructure; actual replication is modest, 2. The overall impact assessment is that the Project has met - or come close to - most of its targets and while it has not been notably successful in health, water supply and the resilience of roads improvements, it has laid a technical foundation and demonstrated practicability for future rural and district development in Tanzania. The overall impact is assessed as between 2 and 3.

Performance of the project

Relevance of objectives. The relevance of the goal at appraisal - of rehabilitation of the region and reversal of the long-term decline in agricultural production - was - and is - high, 4. The specific objective of agricultural development - to improve household food production and incomes in a sustainable manner - was apposite because it implies cost-limited but technically proven answers to problems of declining productivity; it continues to be germaine, with a high relevance, 4. The environmental management objectives of supporting afforestation, control of water hyacinth and promotion of wood saving stoves are considered valid, the proposed backing for the appointment of many more district forestry staff, less so; their relevance is put as substantial, 3. Rural infrastructure did not have a credible objective at appraisal but only a vague statement of intent to support the investments in agriculture and environment. The main aims of the health sub-component can be interpreted as to reduce: malaria incidence and severity; maternal and infant mortality and morbidity; and child malnutrition. These are highly relevant targets, but the relevance of the stated objectives, in the context of the severity of the problem, is modest, 2. The stated desired output for water supply was: farmers access to safe drinking water improved; relevance of this aim then, as now, is high, 4. The implied objectives for rural roads were: to facilitate transport of farm inputs and outputs and access to social infrastructures; and to increase skills and sensitisation of communities for road maintenance; the Project facilitation, monitoring and evaluation component had the main objectives of: strengthening implementation and service delivery capacity of district departments and assisting development of a participatory approach to resource management; relevance is high, 4.

Effectiveness. The goal achievement for the whole Project can be estimated from progress in arresting natural resource degradation, reversing farming productivity decline and measurably improving living standards of beneficiaries. Despite variation in impact of different components, aggregate effectiveness is assessed as reasonable, 2. The agricultural development component has largely achieved its objectives, transforming the predicament of participating farmers - and given the opening scenario of social disruption and resource degradation, effectiveness is high, 4. In environment, the component has had problems of staffing, the less immediate and tangible nature of benefits and higher risks; but operations have been quite successful and effectiveness substantial, 3. The health sub-component has been affected by over-emphasis on equipment supply, some of which proved not readily usable or serviceable; deficiencies in training programme design and coverage; inadequate sensitisation and community mobilisation; and inconsistency in supervision; these factors have reduced the effectiveness to modest,2. Water supply has been subject to change and difficulties in implementation; Project objectives of improved access, low operating cost and viable VWUGs have not been met and effectiveness is assessed as minimal, 1. The rural roads sub-component is close to accomplishing its target for works ; effectiveness of meeting the first part of the roads objective is high, 4; that for mobilisation of local maintenance capability is considered modest, 2. Operations under the Project facilitation component have varied from well managed to still only tentative in application, as in the case of M&E, support of PRAs and logical frameworks and gender equity; effectiveness is assessed as reasonable, 2.

Efficiency. The Project in total has entailed deployment of substantial resources and funds, particularly at start-up. The crux of efficiency is the quantum and likely continuity of benefits from the high level of costs incurred. An outline financial re-calculation of the internal rate of return, based on highly conservative estimates of uptake and benefit, show a calculated IRR of 15%, compared to the 19% at appraisal. The resilience of the IRR is also in line with that found at appraisal; that is, a reduction of benefits of 10% still gives a rate of return of 13%; thus the efficiency of the whole Project is put at substantial, 3. For agriculture, present and prospective expenditure is considerably below budget, partly due to cost containment strategies, and efficiency is substantial, 3. In environment incurred costs are lower than planned, despite physical achievements above predicted levels; efficiency is again substantial, 3. The health sub-component has been characterised by supply of unsuitable equipment and deficiencies in cost-sharing; efficiency is assessed as low, 1. Costs of water supply schemes are on average double appraisal estimates and unit water provision costs are over 50% higher than standards; levels of salary/fee payments for skilled labour inputs appear excessive; and sustainability of water source improvements is not well assured; for these reasons, efficiency is low, 1. Records for roads reveal: wide cost discrepancies between districts for similar works; disproportionate design and supervision costs; limited local labour employment benefit; and doubts about arrangements for repair and maintenance; efficiency is modest, 2. The Project facilitation and monitoring component has incurred excessive costs and still has weaknesses in M&E, in district capacity building activities and in gender; efficiency is put as modest, 2. Overall Project performance as assessed by UNOPS supervision missions reveals only minor problems, ranked 2 on the UNOPS scale, except for initial delays in staffing: in the first year for a range of indicators; in years 2 and 3 for loan covenant compliance, the M&E system and time over-runs; but with a continuing problem, up to 2002, of counterpart funding.

Performance of the partners

The handling by IFAD of the process of formulation and design could have been faster and less tortuous and Project concept and content less top-down, ambitious and complex. During implementation, the experience of the PFMU, government and other partners has been that back-up has been timely and effective; overall, performance is ranked 3. As the co-operating institution, UNOPS has made four annual supervision visits, two with IFAD and two with BSF participation. Supervision has been consistent and generally thorough, but limited in terms of medical and water engineering expertise, lax in oversight of district financial recording and control and optimistic with regard to M&E proficiency and target achievement; UNOPS performance is assessed as adequate, 2/3. Central Government has provided good continuity in Project oversight, policy guidance and donor and ministerial liaison, but co-operation of other ministries has been less conscientious and there has been persistent delay and shortfall in counterpart funding. Regional and District Commissioners Offices and Administrations have played a crucial role in enablement and support of project activities. Major district weaknesses have been stringency of finance and resource availability; at ward and village level there have been rather more problems of competence of staff and lack of resources; overall, government performance is satisfactory, 3. Project management, comprising the PFMU, technical cadre and DPFs, has shouldered the major responsibility for driving forward local interest and participation with dedication and competence. Work planning, budgeting and progress chasing have been reasonable; and recording and reporting timely, albeit accuracy and relevance of data presented have been less than sound. Specific shortfalls in management performance concern: use of local organisations and prior beneficiary contribution to schemes; failure of the input supply credit scheme and delay in finding an alternative approach; lax oversight of district expenditures and individual scheme inputs and costs; delay and drift in M&E; and poor technical/management supervision of water supply. In aggregate, Project management performance is assessed as competent between 2 and 3. The Project has engaged available NGO expertise, particularly in innovations in agriculture and group support, with active co-operation and few problems of overlap. The Project itself has been a major player in the development of CBOs in Kagera, in the form of IPM/IPN groups, SGAs and local committees. The NGO/CBO contribution to support of KAEMP is ranked as substantive, 3. The pre-existence of the Belgian funded Kagera Community Development Project, KCDP was a useful precedent for design of KAEMP. The continuing input by Belgian Embassy staff to KCDP and frequent contact with the BSF regional co-ordinator have had practical advantages: no problems have been reported in management of grant funds and BSF/JP has dealt promptly and positively with requested variations in programmes and budgets. The OPEC Fund has maintained an active dialogue with the PFMU and responded in a timely and efficient manner to technical or administrative communication and withdrawal requests. Performance of cofinanciers is put at 3. The performances of the Project and development partners are summarised in the tables below.

Component/Activity	Relevance	Effectiveness	Efficiency
Agricultural Development	4	4	3
Environmental Management	3	3	3
Rural Infrastructure:
- Health	2	2	1
- Water Supply	4	1	1
- Roads	4	4 and 2	2
Project Facilitation/Monitoring	4	2	2
Whole Project	4	2	3
Partner	IFAD	UNOPS	Govt	Project Managment	NGO/CBOs	Cofinanciers
Rating	3	2/3	3	2/3	3	3

Insights and recommendations

Individual insights of key significance for design and conduct of future projects are:

• the zero year concept: the need for a period of preparation for sensitisation, planning, mobilisation, participant liaison, staffing, training and some procurement pre-Project;
• the gender dimension: desirability of clear identification of gender equity issues and explicit description of interventions/mechanisms to ensure fair sharing and improvement in status of women;
• linkage of components: lack of sufficient, ongoing inter-component linkage, e.g. between agriculture and health/child nutrition, can preclude attainment of the full potential for synergy;
• sustainability concerns: questions of sustainability of Project benefits and of definition of the means and methods of phasing out of projects need to be more comprehensively addressed;
• the terms and conditions for district staff involvement: terms of commitment, accountability, job description, incentives, penalties, and opportunities for training/ promotion, need to be spelt out;
• the importance of communications and flexibility: adequate resourcing of communications and provision of flexibility for project management are necessary pre-requisites of impact;
• the quandary of record keeping and reporting: a new approach needs to be devised to make M&E effective in providing management, process assessment and impact evaluation information;
• in contrast, farm business recording has been a notable and replicable success: many IPM/IPN, FFS and SGA members are assiduous users of farm and household budget planning data; and
• the group and minimal cost approach: for information dissemination/technology transfer, the KAEMP/Farmer Cadre approach can be a guide for transformation of extension provision.

The full set of Mission recommendations by component is presented in the Main Report; those selected as being of particular urgency or concern are set out below. It is recommended that:

• direct farmer/group action to improve marketing of produce be instigated/facilitated as a first step to an increasingly commercialised and profitable farming sector;
• group, SGA, roads, water and health committee and CBO viability be consolidated with further and advanced training and continuing advisory support for a period of at least one year;
• the groups, Farmer Cadres and tree growers that KAEMP has supported be promoted as preferred agencies and (paid) consultants/resource persons for other donor and NGO projects;
• responsibility and resources for water hyacinth control be handed over as soon as practicable to LVEMP and/or to the appropriate regional or district agencies;
  an agro-forestry, soil conservation and land management project be devised, for communal natural resource management, involving village governments and the private sector;
• a selected number of Village Health Committees be trained and coached to implement cost sharing as pilot schemes to give feed back to VDCs for health planning and budgeting purposes;
• VHWs and Traditional Birth Attendants be included in additional training in disease prevention and community mobilisation;
• no planned or additional water supply schemes be constructed until their feasibility has been properly assessed and their design approved by a competent professional adviser;
• as the highest priority for water quality and safety for existing Project schemes, protection, hygiene and sanitation measures to prevent or treat for contamination be introduced;
• the decision to install water quality laboratories in all districts be re-visited, with a feasibility study to include a thorough comparative costing of alternative means to meet district obligations;
• a detailed comparative costing be made of a small sample of Project-assisted schemes to determine accurate unit costs and the efficiency of different modes of implementation;
• a small revolving stock of the normal wearing parts of hand pumps be facilitated at district level, preferably through private sector stockists - or private/public sector co-operation;
• replacement of some of the borehole hand pumps that are unsuitable for the present depth/work of pumping be arranged;
• for roads rehabilitation/construction a combined use of labour-based and machinery technology be made; and division of sub-projects into minor and small contracts be avoided;
• the whole M&E approach, systems, data formats and procedures be simplified and streamlined in line with real local capability and resources;
• district staff and village governments and officers be given continued support in participatory planning and M&E - and in the utilization of the logical framework; and
• knowledge and skills in gender analysis and mainstreaming be further consolidated to ensure institutionalisation and integration in regional, district and community plans, including use of gender sensitive indicators and analysis of gender disaggregated data.

The recommendations indicate the strong need for further consolidation of training and knowledge endowment and capacity building of groups and committees that will stretch beyond the proposed Project completion date. The Mission recommends that the possibility be explored of utilising part of the year 6 funding - and a measure of additional funding by IFAD and BSF/JP if proven necessary by a more detailed costing of the inputs required - for a consolidation phase of at least one further year of limited and targeted assistance to optimise impact and obviate what could otherwise be a certain loss of development impetus. This extended input would concentrate on the further training, guidance and management support of groups, SGAs and committees on whose competence the sustainability of Project benefits will largely depend. It would also encompass additional support for districts, wards and villages in planning and development implementation processes. IFAD and BSF/JP should consult with PO-RALG and the Regional and District Administrations, probably in the context of the Agreement at Completion Point, to formulate an appropriate follow-up programme.

---

The Interim Evaluation Mission comprised: Dr Fatima Mohamedali, Public Health Specialist; Michel Van der Stricht, Rural Infrastructure Consultant; Ms Joyce Nyoni, Sociologist; Clifford Tandari, Economist and Local Resource Person; Ms Sarah Mader, IFAD Associate Evaluation Officer and Jim Semple, Team Leader and Agriculture/Environment Specialist. Mr Ashwani Muthoo was the lead OE Evaluator.