Tihama Environment Protection Project (2003) - IOE
Tihama Environment Protection Project (2003)
Interim Evaluation
Introduction
The objectives of the interim evaluation were to assess: (i) the achievements of the project so far and its impact on the target groups in relation to the original design; (ii) the extent to which the project has achieved its objective of developing a sustainable framework for natural resource management that can be replicated in other areas of the Tihama; (iii) the future options for IFAD and Government cooperation in the Tihama; and (iv) to derive lessons from this experience for the benefit of similar interventions elsewhere.
The evaluation involved a review and analysis of project documents and discussions with government officials and extensive interactions with project beneficiaries. Fieldwork was participatory with Yemeni counterparts involved throughout. Fielded in the first semester of 20021 the evaluation mission covered 25 of the 47 project villages and held extensive discussions with over 360 beneficiaries, both men and women on the basis of pre-tested open-ended questionnaire. The mission met senior staff of the Ministry of Agriculture and Irrigation (MAI) and the Ministry of Planning and Development, and had wide covering discussions with the staff from the following agencies engaged in project activities: (i) the Tihama Development Authority (TDA), a semi-autonomous body under MAI; (ii) the Cooperative and Agricultural Credit Bank; and (iii) the Agricultural Research and Extension Authority.
At the end of the fieldwork the mission discussed its initial findings with the Chairman and staff of TDA and briefed the Governor of Hodeidah. The mission prepared an Aide Memoire which presented the mission's preliminary findings and conclusions in a wrap-up meeting held in Sana'a, chaired by the Deputy Minister of Agriculture and Irrigation and attended by members of the Core Learning Partnership (and other representatives of the implementation agencies). The draft evaluation report was shared with the CLP members and their comments taken into consideration to the extent possible. A number of lessons in key areas of project design and implementation were identified by the evaluation and these form the basic content of this agreement. The lessons relate to: (i) project design and supervision; (ii) beneficiary participation and sustainability; (iii) monitoring, evaluation and impact assessment; and (iv) provision of credit.
Evaluation approach
The objectives of the interim evaluation were to assess: (i) the achievements of the project so far and its impact on the target groups in relation to the original design; (ii) the extent to which the project has achieved its objective of developing a sustainable framework for natural resource management that can be replicated in other areas of the Tihama; (iii) the future options for IFAD and Government cooperation in the Tihama; and (iv) to derive lessons from this experience for the benefit of similar interventions elsewhere.
This was the first evaluation to use IFAD's new common framework of evaluation, which has a greater emphasis than previously on impact assessment. The evaluation criteria used are: (i) Rural Poverty Impact covering six domains of impact and one criterion each for sustainability and replicability; (ii) Performance of the Project consisting of relevance of objectives, effectiveness and efficiency; and (iii) Performance of the Partners including IFAD, the Cooperating Institution, Government and its agencies, NGOs/CBOs and cofinanciers (if any).
The evaluation involved a review and analysis of project documents and discussions with government officials and extensive interactions with project beneficiaries. Fieldwork was participatory with Yemeni counterparts involved throughout. The evaluation mission visited Yemen in February/ March 2002. The fieldwork covered 25 of the 47 project villages and held extensive discussions with over 360 beneficiaries, both men and women on the basis of pre-tested open-ended questionnaire. The mission met senior staff of the Ministry of Agriculture and Irrigation (MAI) and the Ministry of Planning and Development, and had wide covering discussions with the staff from the following agencies engaged in project activities: (i) the Tihama Development Authority (TDA), a semi-autonomous body under MAI; (ii) the Cooperative and Agricultural Credit Bank; and (iii) the Agricultural Research and Extension Authority.
At the end of the fieldwork the mission discussed its initial findings with the Chairman and staff of TDA and briefed the Governor of Hodeidah. The mission prepared an Aide Memoire which presented the mission's preliminary findings and conclusions in a wrap-up meeting held in Sana'a, chaired by the Deputy Minister of Agriculture and Irrigation and attended by members of the Core Learning Partnership (and other representatives of the implementation agencies).
Poverty and IFAD strategy
Yemen, which covers 537 000 km2 in the Southwest corner of the Arabian Peninsula, has a population of about 18.3 million and suffers from severe poverty2 because of (inter alia) a poor resource base and a high rate of population growth (3.5%). Poverty affects 30-40% of the population but over 80% of the poor live in rural areas. IFAD has supported 15 projects in Yemen with loans totalling over USD 136 million. Four of these projects are under implementation, while a sixteenth has recently been appraised. IFAD's strategy in Yemen focuses on the mobilization of resources in support of agricultural and rural development activities designed to assist the poorest farmers, entrepreneurs and rural women. A consistent theme has been institutional strengthening as weak institutional capacity has been recognized as limiting project implementation.
Tihama
The arid Tihama plain is the country's most important agricultural area, consisting of sand plains and dunes interrupted by wadi3 flood plains. The Tihama covers an area of about 22 000 km2 with slopping land about 30-60 km in width and extending along the Red Sea to the west and up to the foot of the mountains in the east. Eight wadis intersect the plain and collect run-off from the highlands to the east. The run-off makes its way through incised valleys to emerge on the coastal plain, where the water is used for spate irrigation and recharges the alluvial aquifer (the largest in the country) for urban and domestic water supply and irrigated agriculture. Rarely, small amounts of runoff reach the Red Sea.
Main design features
Project design rationale. Wind erosion, sand dune encroachment and over abstraction of groundwater were jeopardising previous agricultural investments in the Tihama. The region offered opportunities for sustainable agricultural development and an improved standard of living for the rural population if: (i) the threat of sand dune encroachment could be minimized so that the land and water resources could be managed in a sustainable way with the full and active participation of the beneficiaries, including women; and (ii) the management capacities and monitoring and evaluation procedures of the institutions could be strengthened to ensure effective implementation of the project activities and develop replicable models for sustainable environmental protection.
Project area and target group. The project area was specified as the 47 villages and their farmland that lie within 2 km of the contact zone where the inter-wadi sand plains and the wadi flood plains meet in Wadi Siham and in Wadi Zabid. The contact zone is about 70 km long, with around 35 km in each wadi. In this zone large areas were under an immediate threat from sand dune encroachment, groundwater resources were being over-exploited in many places and large numbers of households were amongst the poorest in the Tihama. There were 7 100 households in the 47 villages with a high concentration of sharecroppers and farm labourers. Beyond the end of the life of the project, a total of 20 600 families was expected to benefit from the stabilization of the sand dunes. Cereals, forage and cotton followed by fruit and vegetables are important crops, while livestock provide about 20% of the agricultural output.
Objectives. The project has the overall goal of contributing to the Government's programme for improving the standards of living of rural people. The primary objective is to identify and demonstrate, by implementation on a limited scale, appropriate and replicable methods for the management of natural resources to support sustained and increased agricultural production. The project's specific objectives are to: (i) prevent further encroachment of sand dunes on to farming land; (ii) increase water use efficiency for cropping and livestock in the areas most threatened by encroachment of sand dunes; (iii) increase livestock productivity; (iv) improve women's literacy, family health and nutrition standards; and (v) improve the capacity of the Tihama Development Authority to plan, implement, monitor and evaluate the project and other donor and Government financed projects.
Project components. To achieve these objectives the project has four components. The Land Conservation component (47% of project costs) would stabilize sand dunes using mechanical and biological methods, so providing an environment that would enable villagers to plant on-farm shelterbelts and would monitor the speed of sand sheet and sand dune migration. The Water Conservation component (11% of project costs) would improve existing irrigation systems through the provision of credit to farmers for the purchase of improved technology and evaluate and develop appropriate irrigation technologies and monitor water levels. The Support for Rural Women component (17% of project costs) would provide assistance for livestock production (extension, veterinary services and credit), literacy teaching, health education and services and extension services for household food production and processing. The Management Support component (25% of project costs) would enhance the technical and managerial capabilities of the Tihama Development Authority, including monitoring and evaluation procedures, the provision of a national level liaison officer and support the formation of farmer cooperatives.
Over a seven-year implementation period, the estimated project costs was USD 11.7 million and the IFAD loan USD 9.8 million with the Government contributing USD 1.8 million and UNDP 86 000 (this did not in practice materialize). The long project design period, which started in 1990, culminated in Executive Board approval in April 1993 and the loan became effective at the end of 1995. The loan closing date is 30 June 2003.
The Tihama Development Authority is responsible for implementing project activities (except for the credit and research activities) through its existing structures, with a full-time Project Manager responsible for coordination and follow-up. The M&E unit is to undertake: (i) input/output monitoring; (ii) ongoing evaluation; and (iii) impact evaluation. The Cooperative and Agricultural Credit Bank is the channel for providing project-financed loans under a Subsidiary Loan Agreement with the Ministry of Finance. The eligibility and security criteria for the provision of project loans are the same as those the Bank was using under the IFAD Agricultural Credit Project in the Tihama. For research the Tihama Development Authority and the Agriculture Research and Extension Authority were to operate according to a Subsidiary Agreement, which was to include a planned programme of collaborative research.
Beneficiary participation was identified as essential for the sustainability of project activities and in the absence of grassroots organizations the project was to support the establishment of cooperatives within the project area with the assistance of the Hodeidah branch of the National Cooperative Union under a Subsidiary Agreement with the Tihama Development Authority. The cooperatives were to eventually manage the project investments in land conservation.
Project implementation
Changes in design. Implementation of the project has been characterized by flexibility in respect of the original design with some significant changes, which can be summarized as follows: (i) the size and scope of the credit subcomponents were increased; (ii) there were changes to the 47 project area villages, some of which were logical as the villages excluded did not have a sand problem; (iii) the UNV technical assistance was replaced by five experienced long-term specialists from Sudan reportedly at a comparable cost per person month; (iv) the proposed construction of the fore-dunes was dropped following the recommendations of the technical assistance and a consultancy study; (v) a major programme of short-term overseas training for staff was introduced at a cost of over USD 800 000; (vi) Mesquite (Prosopis juliflora) was replaced by a range of other species for planting in the sand dune stabilization belts; (vii) the arrangements for introducing beneficiary participation through the establishment of cooperatives were not pursued; (viii) at the request of the Governor of Hodeidah the project introduced a sand dune stabilization belt to protect the town of Hodeidah from sand encroachment, part of which is irrigated by sewage water; (ix) the proposed appointment of a liaison officer at national level was deemed unnecessary and dropped; (x) project allowances for Tihama Development Authority staff working on project-financed activities were introduced to improve staff motivation with the cost reimbursed from the IFAD loan. Finally, Schedule 2 of the Loan Agreement was revised in September 2000 to take account of the expected changes in expenditure under the various loan categories and the inclusion of an expenditure of USD 300 000 for the disposal of toxic waste in Wadi Surdud (i.e. outside the project area).
Progress. Despite the two-year delay in effectiveness and slow start to project activities 99% of the loan had been disbursed by March 2002, more than a year before the loan closing date. In addition, to the remaining balance in the Loan Account, the Government has allocated the equivalent of about USD 294 000 for project activities, excluding staff salaries in 2002. Project records do not enable a comparison of project expenditure by component with the appraisal estimates, an important monitoring tool, as the need for such records was not specified in the project design. However, data on physical progress, including comparisons with the project targets, is available.
Summary of achievements and rural poverty impact
Physical achievements. The physical achievements of the project in comparison to appraisal targets are impressive and much experience has been gained during implementation and should be recorded. This can be seen in the length of sand dune stabilization belts established, the associated infrastructure provided (wells, irrigation infrastructure and access roads) and the area protected by on-farm shelterbelts and others. Progress in physical achievements and comparison with appraisal targets were produced on a regular basis by the project and documented by the supervision reports. The IE report (Appendix 2, Table 2) regrouped these data to provide a summary whenever possible on achievements by component. In general, the project must be commended for achieving physical appraisal targets and sometimes surpassing them.
Sand dune stabilization. Despite these favourable achievements, many of the sand dune stabilization belts are perceived by farmers as being too far away to offer effective protection to the village and/or the trees are still too small for their effect to be felt, as many of the stabilization belts have only recently been planted. In villages immediately threatened by the sand dunes, the benefits of protection are more perceptible, but some of these belts suffer from localized gaps reducing their effectiveness.
In addition, there is a serious concern over the sustainability of the investments in sand dune stabilization and hence their replicability because of the current lack of ownership and commitment by the surrounding communities to manage the physical assets built by the project. A main reason for this is that local communities have not appropriately participated in site selection and were not adequately mobilized from the beginning of the process. Sustainability is also unlikely due to the continuing need to irrigate the trees after the end of the project and concerns about the future availability in many places of suitable groundwater for this purpose.
On the other hand, project achievements in establishing the on-farm shelterbelts are highly valued by farmers as they reduce the damaging effects of wind on growing crops while the water used to irrigate the fields they protect sustains their growth.
The monitoring of winds and sand movements was delayed and the subsequent management of the operation was inadequate. Similarly, the lack of monitoring of water levels in the wells is contrary to what was expected at appraisal. As a result the planned monitoring activities have yet to contribute to the design of the sand dune stabilization measures and to their management.
Water conservation. The project has made little contribution to water conservation and has not developed any replicable model for this purpose. The Bank disbursed only 22 loans in the project area for PVC pipes. The project did not test alternative irrigation technologies and project financed research activities were hampered because the cooperation between the Project and the Agricultural Research and Extension Authority did not materialize as planned, despite the agreement signed in 1999. As a result the involvement of the Agricultural Research and Extension Authority was minimal. Many of the research results are only indicative and subject to interpretation due to a lack of equipment and statistical analysis. Of interest is the use of sewage water to irrigate trees in part of the Hodeidah Greenbelt, although a build up of salinity, an accumulation of heavy metals, nitrates and phosphates and ground water contamination may create problems in the future.
Support for rural women. The Tihama Development Authority is to be commended for establishing a team of female extension workers, for recruiting and training 22 midwives and for the provision of domestic water supplies for project villages. The project has addressed the needs of poor rural women through its interventions in the area of drinking water, health services, literacy classes, animal production and health extension. In particular, the provision of domestic water supplies and the literacy classes have had an appreciable impact on women's workload and self-confidence. However, the project has had only a modest impact on women socio-economic status, due, among others, to the very limited outreach of project credit activities to the rural women, and there is a large element of double counting in the outreach figure reported by project staff. Only the few women who received livestock loans in kind from the project, most of which were given to the most vulnerable households, appear to have realised a substantial increase in household income and productivity.
The above-mentioned achievements are under threat because the project did not invest adequately in evolving sustainable or replicable models for the adult literacy activities or the provision of health services. In addition, the work of the female staff is hampered by inadequate provision of planned inputs (e.g. transport, materials, training etc.), limited leadership in the area of support for rural women, and the payment of lower staff incentives to female staff than to the male staff. Project activities in land and water conservation and support for rural women have yet to have a significant impact on food production and security in the areas adjacent to the sand dune stabilization belts through increased agricultural production or sustained employment generated by project activities.
Rural credit. CACB has provided 743 loans for a total amount of YER 116.1 million (USD 0.69 million) in the project area4. Of these 22 were for water conservation and 100 were for women, much less than anticipated at appraisal. The poorer households are reluctant to apply for CACB credit due to their inability to meet the collateral requirements, the high transaction costs, the lengthy approval procedures and the high interest rates. Women have had little access to CACB credit due to the low ceiling for collateral free loans and the requirement of land collateral for larger loans. Women do not generally own land and as such a woman cannot fulfil this requirement without the support of her husband or another male household member. The same is true of sharecroppers who can qualify for a loan only if the landowner is willing to offer his land as collateral. Only 6% of the households in the project area have obtained loans from CACB under the project (see also paragraphs 45 and 46).
Management support. Staff of the Tihama Development Authority have benefited from the presence of the technical assistance between 1998 and 2001 and should now be able to handle the technical aspects of sand dune stabilization without significant outside assistance. The introduction of staff incentive payments has helped motivate staff and contributed to the good implementation of physical works. However, project implementation has not been without managerial problems5 and the Tihama Development Authority did not recruit the planned management and finance technical assistance. The M&E unit lacks resources and has not received the inputs envisaged at appraisal. It produces regular reports on physical achievement but no assessments have been made of the social and economic impact of the project on the beneficiaries. The M&E function is not yet an integral part of management nor has it functioned as an appropriate management support. The project has financed the construction of a large training hall but it is not yet in use due to a dispute with the contractor, while the large investment in overseas training seems to have had little effect on the subsequent performance of the trainees.
Community participation. Participation, as envisaged at appraisal, has not occurred and there was never any agreement between the National Cooperative Union, or any similar agency, and the Tihama Development Authority. The villagers do not know why the sand dune stabilization belts were located on their present sites and were not consulted on the selection of tree species. Stabilization sites, determined by the project on the basis of technical considerations, do not seem to reflect village level perception on desired sites for effective protection from sand encroachment. There were little local level discussions regarding the subsequent management of the physical assets or ownership of the trees. The project has involved communities to the extent that farmers on whose land the trees are planted have to agree prior to the start of planting.
Employment created by the project has been highly valued in the project area and contributed to household income in the short run. While landowners are likely to have benefited more than the others in securing project employment, the landless, tenants, sharecroppers and daily labourers have not been excluded. When employment is provided on a rotational basis, as has occurred in several places, there is an element of equity in the distribution of project benefits from employment. However, the project has had a very limited impact on social capital formation or grass roots institutional strengthening partly because the project did not implement, consistently and timely, the beneficiary participation aspects included in the project design. As a result, the communities have passively received most of the project benefits.
Sustainability. The project management, on the basis of supervision recommendations, has only recently promoted the establishment of Environmental Protection and Development Associations (EPDA) at the village level to manage and finance the sand dune stabilization belts when project assistance finishes. Eight such associations were established by March 2002. While no doubt a commendable effort, the establishment of these associations seems to be too little, too late. These associations do not appear well grounded at the village level and seem to be an initiative of the project rather than a genuine grass-roots level initiative. They do not enjoy any autonomy and the supervisor hired by the project to oversee the work on the sand dune stabilization belts is still seen as the key person responsible for all decisions regarding the management of the trees. The seven-member committee do not appear empowered regarding management decisions is not playing a proactive role in making those decisions. While some contributions are made by members on a limited scale, there is little indication that the associations will be able or willing to finance and manage the tree plantations after the end of the project. They have yet to establish any formal system for Operation and Management of the wells and water use and have yet to demonstrate that they have the requisite management capacity. The evaluation found little evidence that these associations would effectively contribute to building village level institutions capable of managing the sand dune stabilization infrastructure. This casts serious doubt as to whether the trees will survive without continuing financial and management support from TDA. The project has limited expertise in participatory approaches but has not sought the support of available expertise in this respect (e.g. local NGOs).
The Tihama Development Authority has not so far provided a comprehensive model for sand dune stabilization that is socially and economically sustainable and hence potentially replicable. The project should determine the actual costs of operating each of the wells so that a realistic estimate can be made of their financial viability. Households in some villages are reportedly paying YER 100 to 300 per month and a preliminary estimate by the evaluation indicated that the annual capital and running costs of a well and associated infrastructure may be over YER 500 per household per month, and so may not be affordable. The Hodeidah Greenbelt can only be sustained by involvement of the Tihama Development Authority or local government, as there are no associated villages to participate in the continued operation and management of the associated irrigation facilities or to protect the trees.
The project has not, as yet, had any impact on the policy or regulatory framework in the Tihama or more broadly at national level. The methods of land and water conservation that the project has implemented have not led to the development of policies that might be followed elsewhere in the Tihama or in other parts of the country, which might benefit from similar investments.
Overall Poverty Impact. With the change to the original list of villages, the population in the project's 47 villages exceeds 12 000 households (a 69% increase), of which 30% resides in the small town of Al Marawah. The mission's analysis indicates that a majority of the households in the project area fall within the IFAD target group. While some project activities clearly benefit the poor, as mentioned above, the poverty impact of the sand dune stabilization efforts is still uncertain. The project could have targeted its sand dune stabilization activities better to poorer villages without sacrificing the technical aspects of stabilization. For example, Bait-ul Hadi could have been included instead of Al Marawah. In addition, lack of real community level participatory approaches from the beginning of the stabilization processes is likely to limit the potential impact of sand dune stabilization on rural poverty and its likely sustainability.
Performance of the project
Relevance of project objectives. The project objectives (paragraph 9) are still relevant to the needs of the target group in Tihama and the country as a whole. First, sand dune encroachment is still a threat to rural livelihoods, while a failure to arrest the decline in ground water levels and increasing salinity will also increase poverty. Secondly, livestock provide a significant proportion of household subsistence requirements and women, as the livestock managers, are concerned with improving livestock productivity as well as with improving literacy, family health and nutrition standards. Improvements are needed and possible. Finally, since the Tihama Development Authority is almost the only agency that is providing any support to the rural communities in the project area an improvement in its performance is one prerequisite for achieving the other objectives.
Project objectives are consistent with IFAD strategy in Yemen (as embodied in its COSOP), IFAD's mandate, the Government's objective of improving the standards of living of the rural population and the Interim Poverty Reduction Strategy Paper with its focus on expanding economic opportunities for the poor in the agricultural sector. While preventing further environment degradation in the Tihama area will benefit the whole population, most of whom can be regarded as members of the IFAD target group.
Effectiveness. The effectiveness of the project in terms of achievement of its relevant objectives at the time of the evaluation is modest. While the project has developed a technical model for the management of natural resources (i.e. sand dune stabilization) there is no evidence at present that this model is sustainable and replicable. The project did not develop appropriate and replicable methods for the conservation of water resources. The well-known technology proposed is being widely promoted by the World Bank Land and Water Conservation Project in ways that are more advantageous for the farmers; hence few project loans have been made for investment in this technology. In addition, the water conservation research has been less effective than it should have been due to a number of shortcomings of the research programme (see above).
The project has gone some way to achieving the objectives relating to improving livestock productivity and improving female literacy and health but achievements were below expectations due to insufficient support provided by the Tihama Development Authority and low likelihood of sustainability. Discussions with the Ministries of Health and Education recently initiated by the project may lead to the continuing employment of the project-financed midwives and a continuation of the literacy classes to enhance the prospects for the sustainability of these investments.
There has been an improvement in the Tihama Development Authority's technical abilities in relation to the planning and implementation of sand dune stabilization measures, but not in respect of water conservation, overall project management, including M&E or its ability to support a programme for rural women and community level participation. Further improvement in management requires the introduction of modern management techniques and will require the provision of additional technical assistance and is beyond the scope of the project.
Efficiency. The absence of project data records showing the costs by component prevents any assessment of the cost effectiveness of the various project components. In addition, the recent vintage of much of the stabilised dunes (tree plantation) prevents a comprehensive ex-post analysis of benefits. A revised estimate of the potential return for the sand dune stabilization component with the mix of crops found currently in the Tihama and project cost data shows that the investment in sand dune stabilization belts should be economically worthwhile, provided: (i) the sand dune stabilization belt prevents sand moving onto productive land immediately; and (ii) the community manages the sand dune stabilization belts when project assistance stops. If the sand dune stabilization belts continue to be managed effectively the efficiency would be high. The analysis shows that about two thirds of the benefits go to the landowners/pump owners, with the sharecroppers and labourers as a group receiving 26-30% and the Government the remainder (5-6%). However, if the communities do not manage and protect the sand dune stabilization belts after the end of the project, which at present appears to be the likely scenario, the investment is not worthwhile (negative returns) and its efficiency is only modest.
Performance of partners
Performance of IFAD. While the project design was well targeted, innovative and identified key issues that were essential for sustainability, it did not provide sufficient guidance and options as to the participatory approaches and procedures that would help to ensure the sustainability of project interventions. In addition, insufficient attention was given to other key implementation issues e.g. procurement ceilings and auditing requirements.
IFAD's role during implementation consisted of following up and taking actions on the issues that required its attention and carrying out a Mid-term Review. There is no evidence from the supervision reports of IFAD taking action in respect of key failures to comply with the Loan Agreement (see below) nor with respect to the issues of sustainability and timely involvement of local communities. Finally, IFAD agreed to the Government's request to use USD 300 000 from the IFAD loan for the disposal of toxic waste in an area outside the project area (i.e. Wadi Surdud) and amended the loan agreement accordingly. This transfer of funds contributed to the lack of resources experienced currently by the project when belatedly it is endeavouring to develop locally based mechanisms for the sustainable management of the sand dune stabilization belts.
The Cooperating Institution (UNOPS). Since Loan Effectiveness, there has been an UNOPS start-up mission and seven UNOPS supervision missions, while IFAD carried out the MTR. Five points should be noted: (i) there was considerable continuity of UNOPS personnel throughout the supervision process which is highly desirable; (ii) the IFAD Project Controller/Country Portfolio Manager participated in the first three missions; (iii) supervision missions were reduced for budgetary reasons from two p.a. to one p.a. from 2000, which was unfortunate because of the continuing need for support (see later); (iv) there was some technical (agronomy) input into the supervision process before the MTR but not subsequently, which is equally unfortunate; (v) beneficiary participation was stressed only starting 2000, which led to the belated establishment of EPDAs; and (vi) mission composition did not include a social scientist/grassroots institutions specialist despite the need for support in this area.
The evaluation assessment is that the UNOPS' reporting was comprehensive, regular and adequate with respect to recording and analysing physical achievements. The supervision no doubt was an important element in ensuring timely implementation and many problems were identified during supervision missions and appropriate recommendations made. The CI and IFAD also have to be commended for the flexibility shown in adjusting project design during implementation. Supervision missions, however, gave overall an over optimistic assessment of the status of the project until 1999 particularly with respect to the performance of the management, institution building and beneficiary participation. Also the monitoring of compliance with the loan agreement was incomplete. According to the supervision missions from 1997 to 1999 the project had minor problems, but performance was always improving, although IFAD classified the project as a problem project in early 1998 due to the low rate of loan disbursement. Follow up on recommendations made to ensure their implementation did not lead to significant improvement in a number of key areas of project implementation as shown by the continuing inadequacies of the M&E, the lack of audit reports, lack of beneficiary participation and some non compliance of loan agreement (see below). UNOPS made no overall assessments of project performance in 2000 and 2001, but based on the low ratings for M&E, auditing, quality of accounts and beneficiary participation downgraded assessment of project performance in 2001 and to a lesser extent in 2000.
Government and government agencies. At the time of project design there were no specific Government policies related to poverty reduction6. Participation of the rural poor in project design was through a socio-economic survey and beneficiary participation survey using rapid rural appraisal techniques. This survey highlighted the need for participatory development and village level organizations.
Tihama development authority. After a slow start the rate of project implementation in achieving physical targets has markedly accelerated such that the loan is now almost 100% disbursed. However, a number of managerial problems were observed. These include: (i) difficulties in coordinating project field activities carried out by the relevant departments of the Tihama Development Authority, as their heads are senior to the Project Manager; (ii) the lack of terms of reference for most project staff, except the Project Manager, which led to staff being often unclear as to their project responsibilities; (iii) perceived interference by senior management of the Tihama Authority in project implementation; (iv) cumbersome procedures for submitting withdrawal applications both within and outside the Authority and delays in their processing7; (v) inadequate draft audit reports for 1998 and 1999; (vi) non-compliance with some key covenants of the loan agreement; and (vi) a lack of delegation by the project management of routine administrative tasks.
The technical assistance has had no apparent impact on the managerial performance of staff involved in project activities as the persistent problems reported in the supervision and Mid-term Review reports indicate. Much of the overseas training had limited beneficial impact, as it was inappropriate for many of the participants, covered too many subject areas in insufficient detail and was too theoretical and/or related to conditions that were unlike those in the Tihama8. At least USD 110 000 was expended on training staff not involved with project implementation. The M&E unit has not played its anticipated role.
Cooperative and agricultural Credit Bank. As per CACB records, credit disbursements under the project were at 1,073 loans for a total amount of YER 170.8 million (USD 1.02 million). However, these included 330 loans for YER 54.7 million (USD 0.33 million) given before 30 September 1999 to farmers of areas along Wadi Siham and Wadi Zabid but outside the project villages, for which use of the loan funds was allowed as a special case. All subsequent loans were made for the project area beneficiaries. The repayment rate for project loans was only 62% at 31 December 2001 and there is an urgent need to improve the timely repayment of the loans without which the sustainability of the project-supported lending activities is in doubt. Although required by the Subsidiary Loan Agreement, the Bank has not set up a single Revolving Fund to account for project loan recoveries and their subsequent use to make new loans in the project area.
The Cooperative and Agricultural Credit Bank collateral based lending prevents it providing loans to sharecroppers and women. The recent reduction in lending rates will have an adverse impact on the Bank's already difficult financial position. A restructuring programme has been formulated to enable the Bank to become an efficient rural financial institution and IFAD has offered to support the implementation of this restructuring programme.
Implementation of recommendations. One measure of the performance of the Government agencies is the extent to which the recommendations of the supervision missions and the Mid-term Review had been implemented by the date of the following supervision mission. While nearly 60% of all the recommendations were either implemented or partially implemented, the proportion decreased with time and was especially low in respect of the Mid-term Review. This last point may reflect the fact that the Mid-term Review tried to address key implementation issues e.g. participation and sustainability with which the project has had difficulty.
Compliance with loan covenants and agreements. While the monitoring of the compliance with loan covenants was not complete, the Government was not in compliance with a significant proportion of those covenants that the supervision missions did monitor. The assessment of the current compliance by the evaluation mission was that the Government was not complying with 30% of the covenants, including key ones relating to M&E and auditing.
Recommendations
Land conservation. In respect of sand dune stabilization the Tihama Development Authority should consider the implementation of the following technical and financial recommendations: (i) produce the planned sand dune stabilization manual; (ii) repair localised gaps in the sand dune stabilization belts; (iii) discontinue financing the development of the Hodeidah Greenbelt with project funds; (iv) use the checkerboard system of mechanical stabilization as fences facing in four directions will provide protection for the trees from the changing wind directions in the Tihama; (v) regard sand dune stabilization as a long-term investment, almost in perpetuity, given the scale of the problem in the Tihama hence allocate sufficient resources to develop sustainable models for this purpose and to support the schemes in the short run (see paragraph 51); (vi) develop an approach suitable for the coastal areas, which are the major source of the sand problem and allocate appropriate resources accordingly; (vii) investigate a number of fast growing and productive exotics that have been successful in shelterbelts in Wadi Tuban and elsewhere in the region; (viii) undertake a more detailed biological and chemical analysis of the sewage waters, and determine the most suitable course of action for future use of the waters, including assessing their impact on fisheries and mangroves; and (ix) contract a suitable agency to maintain the climate stations.
The hydrology section of the Tihama Development Authority, which has the necessary experience, should (i) gradually assume responsibility for the climate stations and downloading the data; (ii) establish a series of monitoring points to chart the movement of sand dunes at sites to include those where the mission has made a record and there are climate stations recording wind speeds and directions; (iii) monitor wells throughout the Tihama, including the 45 project wells as a priority to obtain an accurate picture of any changes to aquifer quality, water table levels and discharge; and (iv) TDA or the Government should commission a new inventory of wells and a hydrogeological survey of groundwater conditions in the Tihama in order to update the older studies (last done for the whole Tihama in 1986-88) and quantify the status of present groundwater use and likely future consumption.
Sustainability of current sand dune stabilization belts. With respect to this major issue the evaluation recommends the following: (i) remaining project funds should be devoted to developing, supporting and testing the management capacity of the existing Environmental Protection and Development Associations and establishing them where they do not yet exist; (ii) the project should initiate dialogues with and seek support of the Governor of Hodeidah, the Agricultural Co-operative Union, the newly elected Local Councils to assess the extent to which these organizations can help to support community mobilization at the village level for the maintenance and financing of the sand dune stabilization belts; (iii) the project should seek the services of an experienced and recognized local NGO to help in strengthening the existing associations and in supporting the formation of new ones, whenever possible, while examining the most effective modalities to ensure their managerial and financial autonomy; (iv) the project should establish the actual costs of operating each of the wells and maintaining the sand dune stabilization belts so that a realistic estimate can be made of their financial viability; (v) support to any future sand dune stabilization activities should be conditioned by the success of the project in developing a viable model of community participation that shows that the sand dune stabilization belts will be sustainable after the end of the project.
Water conservation. The donors should consider a unified approach to providing farmers with on-farm water saving technology, since the project's approach is much less attractive to farmers than that used by the World Bank. To avoid reducing groundwater levels, seawater intrusion and a continuing rise in salinity of the water used for agriculture, the water supply for Hodeidah may have to be augmented by desalinised water. There is still a need to undertake a comparison of different conveyance systems and the use of bubblers and drip under farm conditions. All future research activities sponsored by the Tihama Development Authority should be undertaken with full participation of the Agricultural Research and Extension Authority.
Support for rural women. The Tihama Development Authority should demonstrate how its activities in support of rural women can be made effective and sustainable by implementing the following recommendations: (i) provide the female extension staff with two additional vehicles on a full time basis; (ii) introduce gender equity in respect of the staff incentive allowances paid; (iii) provide the refresher training for female extension staff envisaged at appraisal; (iv) discontinue the construction of the livestock demonstration pens; (v) provide a full inventory to the female extension staff of all inputs provided as these staff are unaware of the location of many items reportedly delivered to the villages; and (vi) recruit qualified women staff to provide the leadership required.
The commendable dialogue initiated with the Ministries of Education and Health should be followed-up to ensure the effective use of the extension staff trained in adult literacy, nutrition, family health, and develop a long-term arrangement for the effective deployment of the project-trained midwives. These arrangements should also consider the provision of basic equipment and supplies required by the midwives at the village level and introduce an element of cost-recovery that will ensure sustainable service provision by them at the village level. The project should identify teachers at the village level and help them develop a sustainable model for the delivery of adult literacy and primary education for girls in the villages through the introduction of affordable school fees, etc.
Credit. To ensure that the Cooperative and Agricultural Credit Bank does not face any funding constraint to meet future credit demand during the rest of the project period, the project and the bank should jointly assess, as a matter of priority, what action should be taken to provide any additional funds, if project lending cannot be met from the repayments of existing project loans. To improve disbursements of loans for water conservation and for rural women, the Bank should urgently review its loan eligibility and increase the ceiling for collateral free loans, use informal intermediaries such as savings and credit associations and self-help groups9, and employ female credit officers. To improve the low repayment rates for project loans Bank branches should introduce concerted efforts through follow-up visits and loan recovery campaigns. The Bank should withdraw the instructions to the project area branches (including Bajil, Beit-el-Faki and Hays) to maintain a branch level separate Revolving Fund and instead take steps to set up one consolidated Revolving Fund and to maintain an up-to-date record of recoveries of project loans and their deployment for re-lending. This will also ensure that recoveries of past loans to farmers outside the project area are re-lent to project area villages. Early implementation of the entire restructuring initiative, which IFAD plans to support under the proposed Dhamar Highland Areas Participatory Development Programme, would help the Bank become a sustainable financial institution.
Management support for Tihama development authorityFuture short-term technical and managerial training should be based on a training needs assessment and not exclusively overseas, with outside trainers brought in where necessary and use made of the Authority's considerable facilities. Future project activities need a strong Project Management and Finance Adviser and an M&E Specialist to help introduce and institutionalize modern management methods. The project manager should also have greater seniority to facilitate the coordination of project activities implemented by the departments and sub-regions.
The project should immediately identify key indicators so the Project Completion Report can include an assessment of the impact of the project. In addition, the incomplete training report should be finalized. The M&E unit should be provided with the resources envisaged at appraisal so it can complete these tasks. The Cooperative and Agriculture Credit Bank should provide a socio-economic profile of the borrowers to assess project impact. The Tihama Development Authority should undertake an economic analysis based on site-specific data to establish the economic viability of the sand dune stabilization belts. In future, monitoring and evaluation should be an integral part of the department responsible for project implementation and should function as an effective support for project management and not seen as a donor requirement.
Project design. As the project is of a relatively old design vintage many of the recommendations concerning design are already incorporated in other ongoing projects in Yemen. Project design documents should include the details of the participatory process to be followed, including the arrangements at village level, who will undertake this work; and a sufficiently detailed analysis to demonstrate that the organization(s) to be involved have the capacity and experience to carryout the work involved. Adequate and experienced Non-Government and community-based organizations should be identified from the start and the participatory processes initiated from the very beginning of project implementation.
Project design documents should specify: (i) a requirement for the project to record project costs by component as well as by category of expenditure so management can compare project costs by component with the design estimates; (ii) greater specificity concerning the audit requirements; (iii) a review of the Government procurement procedures to ensure compatibility with the IFAD procurement guidelines; and (iv) cost recovery/sharing arrangements for project-financed services should be included as part of the design.
Project implementation. IFAD needs to review the adequacy of resources to the Cooperating Institution to give the support required during implementation. Two full supervision missions a year are necessary with the presence of technical specialists (including social scientists) when required. Supervision must include a focus on: (i) the implementation of the subsidiary agreements and monitoring compliance with loan covenants; (ii) addressing the issue of beneficiary participation and targeting from the start of the project; (iii) ensuring that the resources proposed at appraisal and needed for implementation are provided10; and (iv) the resolution of issues relating to procurement, delays in submission of withdrawal applications, M&E, auditing etc. In addition, IFAD needs to take prompt action in respect of non-compliance with the loan agreement if the problems identified are not to continue to hamper project implementation as has happened with this project.
Options for future IFAD assistance to the Tihama. One option for IFAD would be to continue to assist the Tihama Development Authority with sand dune stabilization. The evaluation cannot support this option unless the project provides clear evidence in the remaining implementation period of sustainability and economic viability of the completed sand dune stabilization belts (see recommendations made in this respect, paragraph 51). As discussed with project staff, TDA and other partners at the time of the field evaluation and during the wrap up meeting, evidence at that time for such a sustainability were not favourable (the various reasons are stated in this summary and the main report). One of the main reason is that the EPDAs were not formed from the very beginning of sand dune stabilization process and the communities had little real involvement and say throughout the various stages. No clear evidence was found to indicate commitment on the part of local communities to sustain the existing schemes.
Other options for IFAD support based on the analysis of current experience could include: (i) support for rural women in one or more wadis with similar components to those included under the present project's Support to Rural Women with a focus of livestock, including forage production, water supplies, rural financial services, literacy and primary health care; and (ii) support for agricultural services to strengthen extension, adaptive research, rural financial services and animal health and plant protection services targeted at the smaller land and pump owners and their sharecroppers. However, any new initiative for IFAD in the Tihama must involve the potential beneficiaries during the design phase and throughout thereafter and should be consistent with the strategic thrusts outlined in the 2000 Country Strategic Opportunities Paper and other considerations relating to the IFAD lending programme.
1. The mission was organized and supervised by Ms Mona Bishay (Senior Evaluator from IFAD's Office of Evaluation) with the following membership: Messrs Michael Rayner (Team Leader/Economist), Hamdi Eisa (Farming Systems), Andreas Gerrits (IFAD Associate Professional Officer/Monitoring and Evaluation), Bal Godbole (Rural Financial Services/Institutions), Neil Munro (Land Conservation/Sand Dunes) and Ms Maliha Hussein (Rural Sociologist/Gender).
2. According to the 2001 UNDP Human Development Report Yemen has a HDI ranking of 133rd and Gender Development ranking of 131st out of 162 countries.
3. A run-off valley is called a wadi.
4. As per CACB records, credit disbursements under the project were at 1,073 loans for a total amount of YER 170.8 million (USD 1.02 million). However, these included 330 loans for YER 54.7 million (USD 0.33 million) given before 30 September 1999 to farmers of areas along Wadi Siham and Wadi Zabid but outside the project villages, for which use of the loan funds was allowed as a special case.
5. Highlighted in the supervision reports and the Mid-Term Review report and also see Section V.
6. The Government prepared an Interim Poverty Reduction Strategy Paper in December 2000.
7. The national level officer envisaged at appraisal, but never appointed, could have played a crucial role in speeding up the processing of withdrawal applications in Sana'a.
8. An assessment of this training was under preparation by the project but was discontinued due to a reported shortage of funds.
9. A start is being made under the IFAD-assisted Al-Mahara Rural Development Project and similar arrangements have been agreed for the proposed Dhamar Highland Areas Participatory Development Programme.
10. The increased allocation for credit during the parliamentary review may explain why some of the resources originally intended for the support for rural women and M&E were not provided to the detriment of their performance.