A. Key Principles Accountability: For IFAD to be held accountable for delivering a country programme, the COSOP Results Management Framework must be an instrument to enable monitoring and evaluation of the progress in achievement of COSOP objectives, outcomes and milestones over a five year period. To enable evaluation, there must be a baseline, and the indicators in the Framework must be quantified in advance as precisely as feasible. Baselines: The baseline data used to establish the starting point for each COSOP would be gathered in a Baseline Poverty Analysis (described in Appendix V of the COSOP guidelines) during the design of each COSOP. Baseline data would normally be gathered from existing data sources, not from surveys. The baseline data would need to correspond as closely as possible to the strategic objective statements in each COSOP. Surveys would only be organized when there are significant gaps in the existing data sets, and would be subject to resource availability. Selection of strategic objectives, outcomes and milestones: The number of quantified indicators for the Results Management Framework should be keep minimal, and should be limited to key relevant data. The selection of SOs themselves will be heavily influenced by the nature of the ongoing projects in a country programme, given that the Results Management Framework is going to focus on what is going to be delivered on the ground over a five year period. The selection of outcome indicators will by and large be from the list of RIMS level 2 indicators (purpose level statements), while the selection of milestone indicators will be from the list of RIMS level 1 indicators (output level statements). However, as the RIMS indicators are rather limited, there will obviously be occasions when CPMs have to complement them with additional indicators. Quantification of indicators: Once the baseline for each of the COSOP strategic objectives is established on the basis of existing data sources, the expected COSOP outcomes and milestones will need to be quantified. The quantification of the outcomes and milestones in the Results Management Framework would be done through two methods:
This is illustrated in the box below for a country programme with three ongoing agricultural projects. With an SO for “increased access to technology”, the quantification of the associated SO outcome indicator of “technology adoption” would be calculated by adding up the “technology adoption” targets in the agricultural components of the 3 projects already under implementation in the country programme. Annual reporting: Once a year, the CPM would report on progress, against the baseline and the quantified targets. This would be done by aggregating the level 1 and level 2 indicators in the RIMS tables for all ongoing projects, and inserting the aggregated totals into the Results Management Framework. B. Structure of Results Management Framework In order to fit onto a single page, the Results Management Framework has been restricted to five columns. Column 1: Poverty Reduction Strategy targets (or similar). This column would list the key targets in the country’s own poverty reduction strategy (i.e. PRSP or similar) that are aligned to the selected COSOP strategic objectives (SOs). The targets should be as quantified as possible – rather than referring to vague over-arching text on PRSP pillars or themes. Column 2: The COSOP strategic objectives. The second column of the results framework seeks to align the COSOP SOs with the country-level PRSP objectives and targets. This second column would usually include three SOs. More than one SO can contribute to the same PRSP target or a single SO may contribute to more than one PRSP objective – the framework table can accommodate this by vertical overlapping of cells in the adjoining columns. In some cases, linkage with the PRSP may be constrained by lack of detail in the PRSP. In this case, linkages may be limited to higher order objectives – such as agricultural sector growth rather than lower order objectives such as increasing access to land for poor women. Importantly, in situations where the PRSP does not have a rural agenda, there is provision in the results framework for an additional row at the bottom of the table for an SO (or cross-cutting theme) which is not linked to the PRSP. Column 3: COSOP outcome indicators. The outcome indicators relate to achievement of each SO, and the contribution of the SO towards higher level country goals (PRSP pillars). These indicators are similar to purpose level indicators in a project logframe, and indicator information would usually only be available once projects have been implemented and at least reached their mid-point. These outcome indicators could be picked from the list of IFAD RIMS level 2 indicators which are included in the logframes of ongoing projects. Column 4: COSOP milestone indicators. In addition to outcome indicators, this column shows “milestones” – indicators of progress towards the achievement of SOs. Such milestones, also known as “intermediate objectives”, may show outputs (such as operational beneficiary groups) and activities (e.g. training courses). However the number of such indicators shown in the results framework should be limited to 10 to 20 so that the framework can fit on a single page. These indicators could be selected from the list of IFAD RIMS level 1 indicators. Column 5: Specific policy/institutional reform targets related to the SOs (the COSOP policy dialogue agenda). This column lists key policy or institutional issues linked to each SO. These issues may be areas where policy change is needed, and where IFAD activities for the SO may contribute to policy change – either as a result of policy dialogue and advocacy, or influenced by the demonstration provided by a project. These policy ambitions should be worded as detailed as possible to ensure ease of monitoring and evaluation. Furthermore, they should be selected carefully to reflect realistic chances of success, given the fact that the targets will be evaluated. Schematic linkages in COSOP results smanagement framework
COSOP results frameworks require quantified indicators to provide evidence that the Strategic Objectives (SO) are being achieved. Selection of indicators The use of indicators can be made into an elaborate science demanding a major workload. Using a large number of different indicators, however, has no merit in itself. The key to good indicators is credibility – not volume of data or precision in measurement. A quantitative observation is no more inherently objective than a qualitative observation. Large volumes of data can confuse rather than bring focus. It is more helpful to have approximate answers to a few important questions than to have exact answers to many unimportant questions. The fundamental challenge with indicators is to meaningfully capture key changes or “results”. This is accomplished by combining what is substantively valid with what is practically possible to monitor. In development, it is often difficult to make objective and exact observations of the complex development changes we are addressing. Instead, we frequently rely on observations that approximate intended changes. We use indicators that are commonly understood to be closely related, e.g. share of social expenditures in a government budget as “proxy” for poverty orientation of national policies, or proportion of parliamentarians who are female as “proxy” for empowerment of women in national decision-making processes. The following criteria and questions may be helpful in selecting indicators. As a memory aid, the acronym “SMART” summarizes key criteria, asking “Is the indicator specific, measurable, attainable, relevant and trackable?” Specific:
Measurable:
Attainable:
Relevant:
Trackable:
Be sensible and practical in applying these criteria. No one indicator will satisfy all criteria equally well. Ultimately, the choice of indicator is determined through a holistic assessment of validity and practicality. The selection of indicators is an iterative process, building on consultations between programme managers, stakeholders and partners. The process of selecting an indicator takes several steps including brainstorming ideas, assessing each one and narrowing the list (using the criteriaabove) and, finally, making an indicator monitoring plan Note: UNDP has produced some useful guidance on selection of indicators for its results-based management system. Download paper Indicators for Strategic Objectives SOs seek to bring about change, and should be written as “change statements” using words such as improve, increase, adopt and establish. Indicators for SOs should therefore provide evidence that these changes are taking place. Some examples of SOs and SO outcome and milestone indicators are shown in Table 1 below, however these have been left un-quantified. In a real life situation, the SOs and their associated outcome and milestone indicators should all be quantified to the extent feasible. Use of these indicators would raise a number of issues – such as definition of target groups, and what is meant by “secure access” to resources and “community-based”. These issues become important at the project level when the impact of the project will need to be evaluated. Most indicators for SOs can be quantified – i.e. the number of women benefiting can be recorded as can the percentage increase in crop yields. This means that quantified outcomes from the COSOP can be reported – although non-quantified result indicators may also matter – for instance a “greater understanding of the importance of the role of women amongst policy makers” could be the outcome of knowledge management work in support of an SO aiming to improve the position of women. It is important not to confuse SOs with project outputs resulting from IFAD-supported activities, but not involving a response or change in behaviour by beneficiaries or their organisations. Examples of outputs which are not SOs include:
Indicators for Milestones of progress towards Strategic Objectives The COSOP results framework will include an additional indicator column for milestones showing progress towards SOs rather than actual achievement of SOs. These can relate to project outputs (such as number of community groups established) or activities (e.g. number of training courses held). They differ from indicators showing achievement SOs, which aim to capture a change in behaviour or response by beneficiaries to the IFAD-supported intervention. Only a limited number of different indicators should be used for each SO so that the results framework can fit onto a single page. The objective in selecting indicators should be to pick ones that provide good signposts for the progress and achievements of each SO, rather than to measure everything that projects may do. For instance rural infrastructure development may include roads, bridges and other measures to improve transport, but it would only be necessary to show one of these (say roads) in the results framework as an indicator of progress towards the SO of improved rural communications.
|
||||||||||||||||||||||||||||||||||||||||||
 |
 |
 |
