Interim evaluation

Background

The Lao People's Democratic Republic is a poor country, with a per caput GDP ranking at 16 out of 206 according to the World Bank Development Indicators, and 140 out of 176 in the UNDP Human Development Index. Ruled after 1975 by a socialist government, the government implemented more liberal policies from the mid-1980s onwards. It is now following development policies supported and agreed with the World Bank and the IMF.

In the 1990s, poverty in the country dropped from 58% of the population in 1992-93 to 52% in 1997-98. However, the same studies indicate that the distribution of income has become more skewed in this period. Poverty is concentrated in the mountainous regions and in the ethnic minority communities, which largely overlap each other. In Bokeo, poverty fell from 63% to 37% of the population in this period.

Target group, project objectives and components

With a total population in the Province estimated at 105 000 in 1993, the project target group is the 70% of rural households in the area without assured food security. No specific target was set for women, and Appraisal merely stated that 'project-supported development interventions in production and social services will pay particular attention to the promotion of women's interests and benefits.' Ethnic minorities and women became explicit target groups at Mid-Term Review (MTR) in 1999.

Appraisal defined project objectives as follows:

• improved food security through more reliable and increased paddy production;
  
• improved animal health leading to better household food security and increased savings;
  
• more sustainable management and utilization of upland and highland areas;
  
• improved economic and social infrastructure, as well as improved health and educational standards for children and adults;
  
• establishment of appropriate beneficiary organizations for defining and executing village- and farm-level development that are as self-reliant and sustainable as possible; and
  
• improved functioning of government services, particularly in the areas of coordinated development management and administration, and sector-specific technical support to rural people.
  

Excluding the OPEC-financed road, the project had seven components, which were to be cofinanced by IFAD and GTZ, with the following allocations:

• irrigation rehabilitation and development (20%);
  
• agriculture-related group promotion (3%);
  
• technology development (7%);
  
• pilot watershed management (2%);
  
• livestock development (7%);
  
• basic social services (3%); and
  
• institutional support and community development (57%).

National and Provincial Steering Committees were included in the design. At village level, Village Development Committees (VDCs) were to be the entry point for the formation of specific interest groups, and selection of villages and schemes was to give priority to the poorest communities.

Comments on design

The strengths of project design reside in the details of the participatory approach, conformity with government decentralization policies, and the focus on components addressing poor people's main constraints, namely rural infrastructure, the availability of paddy land, and livestock development.

Design weaknesses were its neglect of the problems connected with co-management, the multiplicity of project interventions, particularly in the agricultural sector, and insufficient attention to targeting mechanisms, specifically concerning women.

Main achievements

The project has been working in about 76 villages for over 3 years, with a total population of about 26 000 people living in 4 500 households. Thus about 20% of the project area households have directly benefited from the project. In addition, some villages where the project had no other activities have benefited from the improved road network constructed with project funds, in particular the 29 km of asphalt road built between Houay Xai and Nam Keung in Thongpeung district.

IFAD funds have been used for the construction of 1 health centre, 4 agricultural promotion centres, offices and staff housing in the provincial capital and one district. To date, 15 primary schools have been built and a further 7 rehabilitated. A further 4 educational buildings are either designed or contracted. The impact of these buildings cannot be measured immediately but is clearly assisting an improvement in school attendance, among girls in particular.

In addition to the OPEC-funded 29 km of paved road, a further 38 km of rural roads have been built or upgraded with funds from the IFAD loan, and a further 16 km were under construction. Construction of rural infrastructure plays a significant role in poverty alleviation: road construction has led to more than 50% reduction in transport costs for freight and passengers. It has also increased the village-gate prices of crops (maize prices have risen on average by 27% in the areas reached by project-built roads) and of non-timber forest products (NTFPs) (palm nuts are sold at prices 20% higher in villages along the road, and sandalwood prices are 23% higher).

Irrigation development had been completed for a command area of 452 ha on 7 schemes at a reasonable cost; 20 ha of this area was newly irrigated land. A further 199 ha were under contract. To date, 253 families of the 900 intended at appraisal have benefited from this development, with an average holding per family of 1.8 ha. Yields have increased by about 1 ton/ha on upgraded areas. Benefits from irrigation on the land already developed were estimated to be an increase in rice production of 525 tons, valued at USD 66 622.

With respect to agriculture, the main success has been the introduction of new rice cultivars, which have been widely adopted. A number of fruit trees have been successfully distributed, but the issues of husbandry and marketing have not been addressed. New crops such as mushrooms and ginger have been tentatively introduced in a number of villages, with varying degrees of success.

Targeting has been both a strength and a weakness in this project. The targeting of minority populations and communities has been very successful: 79% of the villages where the project works are populated primarily by minority communities. This success is largely due to the convergence of government and IFAD policies and, insofar as poverty is concentrated in minority communities, has assisted the project in reaching the poor.

Community Development activities have been implemented by the Technical Assistance (TA) team and include adult literacy training, village-level hygiene improvements - including the supply of latrines and the impregnation of mosquito nets - as well as the formulation of Village Development Plans.

There have been over 2 400 village-level participants in training activities, including 31% of women, though the total number of trainees is probably about half, as most trainees have been involved in more than one training course. Training has taken place in crop production (horticulture, mushroom cultivation, fruit tree propagation), livestock husbandry, rural credit, non-formal education, community development, and health. A further 423 participants have been government staff trained in rural credit, education, community development, rural infrastructure and institution building activities. In addition, 10 study tours have taken place in Laos and in Thailand for line agency staff and farmers.

Overall, USD 2 363 050 of the IFAD loan had been spent or committed for expenditure on civil works at the time of IE, though only USD 1 321 020 had been officially disbursed. In addition, USD 873 514 had been officially disbursed in other categories. Assuming that current commitments were fully disbursed, total currently planned project expenditure would amount to over 80% of the loan.

Main weaknesses

The project has been affected by difficulties throughout its implementation. The irrigation component will achieve an acreage substantially below design, amounting to 72% of the total intended area; the number of beneficiaries is likely to be a maximum of 60% of the target.

Serious management problems have dogged project implementation, with lack of cooperation and coordination between National Project Management and the TA team; the latter has been involved in activities that were not intended at design, namely investment funding.

The agricultural component has achieved little, involving a wide dispersal into trials of new crops without attention to marketing. Minimal attention has been given to the important sector of NTFPs. Farmers introducing new crops, particularly fruit trees, have not been provided with the technical support they needed.

No rural credit mechanism was included in design. The revolving fund included at MTR has not been operational due to different views on procedures between project management, the TA team and UNOPS. Among other activities that have suffered from the lack of rural finance, the absence of credit has prevented farmers from clearing new land, and women from setting up better marketing networks for their handicrafts and other produce.

The livestock component has been particularly weak. Livestock vaccination was planned to be a major project activity. Although 73 Village Veterinary Workers (VVWs) have been trained, few of them are actually working, and there have been serious problems with the vaccination programme. Most recently, in 2001, widespread vaccination of livestock was not carried out, apparently mainly because of differences of opinion about its effectiveness between the TA team and project management.

While the project has successfully targeted minority populations, the targeting of women and of poorer people within communities have been neglected. Women's authority in agriculture has been weakened as no women are included in the Water Users' Groups (WUGs) and very few women have been trained, even in subjects which are of prime interest to women, such as livestock. Poorer people have not been given priority for project support within communities.

Community Development activities have been operated entirely through the TA team, and implemented by their staff. As a result, local government Rural Development and Women's Union staff have not developed the capacity to assist communities in mobilization and other community development activities, nor have community members developed a capacity for self-management.
Most importantly, the participatory measures included at design to ensure sustainability of investments have not been implemented. Hence sustainability of project achievements is likely to be weak, given that inadequate mechanisms have been established for village-level management of the investments:

• the WUGs were not fully operational and have not been given the training needed;
  
• no road maintenance mechanisms have been established at community level for the roads built;
  
• VDCs were not sufficiently well trained to implement Village Development Plans; those Plans are not genuinely village-level initiatives;
  
• the district Lao Women's Union and Rural Development staff were neither trained nor equipped to support village-level sustainable decentralized activities;
  
• line agency staffing will become an issue as, contrary to the designed 2 incremental staff, according to the MTR a total of 36 additional staff had been hired by the various departments; and
  
• in addition, after changes decided at MTR, project activities are dependent on loan funds for operational costs, in particular DSA and transport, resulting in further threats to sustainability.
  

The Monitoring and Evaluation (M&E) unit has not used the village profile data collected on the villages at an early stage in the project to consolidate them into an overall baseline study of the project area, nor have these data been updated to reflect change and progress. To the contrary, new varieties of more sophisticated, complex and wordy versions of the profiles have been developed, which add little information and make identification of progress and change due to the project even more difficult to identify.

Finally, project supervision has been inadequate, with insufficient action to solve the project's problems.
Lessons learnt

Project scope. With a planned duration of 7 years and a loan of just over USD 4 million, the project is having difficulty in disbursing the funds provided. There are many reasons for slow disbursement, including:

• development of decentralization and increased role of local government in implementation;
  
• involvement of communities in decision-making for project-financed activities;
  
• natural conditions, which limit civil works during the rainy season; and
  
• relatively low cost of individual subprojects. Although the demands on management and line agency staff time remain considerable, disbursements are consequently much slower.

In view of the increasingly participatory and community-based approach of IFAD projects, a lesson from this project is that size of loans, duration of projects and disbursement schedules need to be adjusted to accommodate the facts that participatory approaches and decentralized mechanisms are slower. However, it should be noted that slower short-term disbursements should be compensated in the long run by improved sustainability of investments.

Decentralization and participation. These processes are linked, but not identical: government decentralization policies involve increased participation of local government staff and can also increase participation at the community level. Both require more lead-in time. In order to avoid some of the difficulties and problems that have occurred in the past, and to ensure the sustainability of investments, it is important to establish systematic beneficiary consultation and participatory mechanisms, starting with the earliest stages of any activity.

Cofinancing is often a major source of implementation difficulties. With respect to cofinancing with GTZ, this project is an extreme example of the situation, with what was originally intended to be a TA input having become a separate project (with different accountability structure, its own investment financing, etc.), i.e., parallel financing.

Technical Assistance and co-management issues. A major lesson of this project is that any TA should be managed within the framework of the investment project, not separately. TA personnel (whether short or long term) should work within the framework of a single project management with a single project manager. Issues related to co-management and technical assistance deserve serious research. In a situation where no improvements can be achieved, structural change should be chosen in preference to 'patching-up' efforts.

Supervision. The role of supervision is to ensure that projects are implemented according to the Loan Agreements and the Appraisal documentation. This is both a supervisory and a supportive role. Supervision should help project management to deal with constraints and problems and provide the necessary support. Supervision of this project has been inadequate, not providing enough support, especially since 1999.

Rural infrastructure. Rural infrastructure investments can be a major support to rural development and can assist poverty alleviation. This is the case for roads, schools and also small-scale irrigation systems. If their location is determined with good targeting, these are investments that can help IFAD fulfil its mandate of reaching the rural poor and reducing poverty.

Agricultural development. Insufficient household rice production is the prime indicator of poverty in Laos. Thus, helping increase rice production is the most effective means of influencing household food security. Project focus on this sector was the correct strategy and should have remained the main project activity; diversification into a variety of other crops and agricultural interventions should have been delayed till the primary - rice production - objective had been achieved. The main lesson from this experience is that priority should be given to interventions that are known to successfully alleviate poverty effective, at the expense of innovations whose effectiveness remains to be proven.

Sustainability. Sustainability is the most fundamental criterion of success of a project, and should be constantly and consistently the focus of project designers and implementers. Insofar as beneficiary participation is the main assurance of sustainability, it is essential that this participation be considered a priority in implementation. No physical works should be implemented without prior establishment of participatory beneficiary institutions (be they formal or informal) to design, construct, manage and maintain them.

Recommendations

Concerning the inclusion of Bokeo Province in the proposed Community Initiatives Support Project

In view of the low absorption capacity of Lao PDR for rural development funding, as demonstrated in the Bokeo Food Security Project, with an average disbursement rate of USD 500 000 per annum, and of the demonstrated difficulties of cofinancing, the Evaluation strongly recommends that the Community Initiatives Support Project (CISP) should:

• adjust its overall size according to proven disbursement capacity; and
  
• only include Bokeo if, by the end of the Appraisal mission (mid-November 2001), an open and clear agreement has been reached between GTZ, the State Planning Committee (SPC), District Planning Committee (DPC), Project Support Office (PSO) and IFAD concerning the details of the use of GTZ funds in the period 2001-04. The agreement should include details of funds allocated per activity per district per quarter. All parties concerned are to be fully informed of the details of GTZ fund allocations according to a format similar to that used in appraisal cost tables. A new organigram must have been prepared and agreed in which the Provincial Steering Committee (PSC) and DPC play an increased role of supervision and management, and all staff of the GTZ TA Team report to a single person, the National Project Director, who is responsible for coordinating the project implementation on behalf of the DPC, the PSC and the Provincial Government.
  

Only the long-term and general recommendations are included below. A number of immediate recommendations for the design of Phase 2 and for the remaining period of Phase 1 can be found at the end of the Main Report.

Strategic to Government

Government should address the issue of its civil service payroll. Government needs to decide which services it is to provide, and make available the funds necessary to pay reasonable salaries to the staff who perform these services. Adequate provision for running costs (transport, DSA, etc.) must be included routinely in annual budgets. Without these, no investment will be sustainable.

Decentralization policies are making new demands on local government staff and on community-level leaders. These people have not been trained to fulfil their new tasks. To assist this process, IE recommends that:

• village leaders be given training in social mobilization and participatory development;
  
• local government staff be trained in the relevant management, participatory and accounting skills so that their way of working and work culture will be changed and re-oriented gradually towards providing services to the clients in a demand-driven and participatory manner;
  
• an adequate number of qualified staff be re-deployed from the central and provincial level to the appropriate regional and district levels to ensure that the necessary skilled staff are available in the rural areas; and
  
• to develop accountable systems and procedures to implement the government policy on decentralization, such as contractual arrangements between the provincial governments and the public service providers using public funds for implementation of development project activities.

Once trained, district- and village-level staff should be given more responsibility and authority with respect to financial management of Government and Project budgets. These new responsibilities and power need to be accompanied by effective procedures of financial reporting, monitoring and supervision to ensure accountability and transparency. The VDCs should be allocated some budgets, to be used at their discretion.

While government policies already favour supporting women, to improve the effectiveness of these policies it is necessary that government implements the following measures, among others:

(i) systematically issue land and housing titles in the names of wife and husband for couples;
(ii) promote increased representation of women in key decision-making positions at all levels, and recruit more women in the staff of the project management and line agencies;
(iii) provide adequate resources to ensure that ethnic women have equal and adequate access to education, family planning and health services;
(iv) develop procedures to involve women in participatory planning;
(v) provide gender awareness training for all its staff;
(vi) give the Lao Women's Union (LWU) a leading role in poverty alleviation programmes; and
(vii) include gender-disaggregated data in reporting systems of all levels of government institutions.

Government would find it beneficial to review the terms of its relationship with GTZ. Regardless of the fact that GTZ provides grant funds, government has the right to demand transparency concerning the use of these funds and take an active role in the distribution and allocation of funds available for the country, as well as the working and reporting procedures between the TA team and the national counterpart.

Participatory mechanisms at the local government and community level are essential. They are, in the new century, routinely well designed in development projects. Participation would be strengthened by formulating enabling legislation on rural user groups (irrigation and drinking water, roads, etc.). Similarly, participation for land use activities would be improved through the systematic issue of land certificates for existing farmers, who would benefit from public investment, such as potential irrigation schemes, which should ensure inclusion of women among certificate holders. This would increase the incentive for them to participate effectively in design, construction and operation and maintenance (O&M) of the irrigation schemes and other improvement activities.

Government should give more attention to the positive features of minority cultures and livelihood systems and ensure that they are protected in development activities, while involving minorities in economic development.

While government policy to reduce shifting agriculture is understandable and reasonable, in practice there are few economically viable alternatives for upland farmers. The stabilization of their agriculture has reduced fallow cycles, no alternative sustainable sources of income have been created, and insufficient lowland irrigated land is available for those who need it. Therefore, Evaluation recommends that the policy towards stabilization of shifting agriculture be implemented more gradually, taking into consideration the availability of alternative sources of livelihood for the farmers.

Now that considerable experience has been developed of village-level animal health services, the Ministry of Agriculture and Forestry (MAF) should conduct a study on the field experiences of the pilot programmes of VVWs in the different projects in the country. The study would be designed to issue guidelines for the future development of village-level veterinary services on a user-pays basis in Lao PDR.

Strategic to IFAD

IFAD should, through a detailed study, review the advantages and drawbacks of cofinancing, and possibly reconsider its strategy concerning cofinancing. TA is necessary for many projects, and IFAD should develop mechanisms to ensure that it fulfils its role precisely and does not divert the project from its intended course. Using IFAD loans for financing TA should be considered, as well as other sources of TA financing, to ensure that TA fulfils the role defined at design and according to the implementation needs of projects.

IFAD should seriously examine its mechanisms for supervision of projects, both with respect to frequency and to content. Supervision must ensure that projects follow the spirit as well as the letter of appraisal documents and should play a more active and forceful role in project implementation. The frequency of supervision missions should be determined by project status and could be as often as three times a year for the more problematic projects, as was the case for most of the time in Bokeo. It should routinely be twice a year (one full supervision and one follow-up mission in countries where the cooperating institution is executing more than one project).