Download PDF Version (413 KB)
 |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This Annex summarises 34 methods you might find useful for specific M&E tasks. For ease of use, the methods have been grouped in seven categories:
Each method is briefly explained in terms of purpose, steps and application tips. As these methods are only brief descriptions from longer texts, please refer to the original texts for additional information (see Further Reading). Note that each method can be adapted and mixed with other methods to suit your needs. See Section 6 for more thoughts on information gathering and management. You can also create your own methods. For instance, in Zambia, staff of a drinking water project launched an essay contest in different high schools in order to understand youth’s perceptions and assessment of the project. This method ended up providing information that was not being obtained by other means. The essays revealed that, in many cases, children were being asked to help dig wells to satisfy the project’s volunteer labour quota demand. This prevented them from attending school, an effect that was not intended by the project. With this information, staff were able to rethink how to organise project implementation to avoid this negative effect. Any method can be used in two ways to understand change. Option 1. It can be applied regularly, as a monitoring sequence, to gain insight into trends. This requires creating a starting point, or "baseline" of data (see Section 5.5). Subsequent applications of the method can be compared to the baseline to identify change and try to understand its causes. Option 2. It can be used retrospectively to inquire about change in the project area. This option takes the current situation as the starting point and asks people to describe how the situation used to be, for example, three years ago. While it does not make use of an independently assessed baseline it does aim to compare changes over time. Because it relies on people’s memory, this use is only appropriate if you do not need high levels of proven precision for the data.
A census, which is a full count, is often not feasible for gathering data from the entire population that you are interested in studying. The group may be too big or time, resources and funds too limited to carry out a census. In these cases, you need to select a sample that is as representative as possible of the full population in order to make conclusions about characteristics of the whole population. Therefore, some statistical tools are needed to determine how representative your data are, and thus how reliable the information coming out of your study is. How you choose a sample influences the quality of the final results of the M&E study. If your sampling method is biased or your sample too small, then your M&E results will be less reliable and perhaps even invalid. If you choose sample-based M&E, three factors in particular need to be considered that affect both the methods you choose for the M&E work and the validity of your findings. (More details about such factors can be found in Casley and Kumar (1988) in Further Reading.)
Purpose: To produce a sample, without any prior knowledge or consideration of particular characteristics, that can be considered to be representative of the primary stakeholders being affected by a project intervention. From an M&E perspective, the sample is needed to guide the use of information-collection methods. How to: 1. Start by identifying, naming or numbering all the units in a population from which you want a sample (e.g., villages, houses, people, families), so that every unit has an equal chance of being chosen for the sample. This is the act of making a sampling frame. 2. From the sampling frame, choose who will actually be selected for the final sample by using one of two basic random sampling methods.
Tips on use: Random sampling is more often used in larger-scale M&E analyses than non-random sampling (see Method 2). However, random sampling is not always possible or practical if, for instance, there is not enough time to make up a complete list of the information needed. Existing census records, electoral lists, telephone books or other records should be sought out and used whenever possible – but be aware that these may not always be accurate. However, Casley and Kumar (1988) warn against a hasty decision to opt for non-random sampling simply based on resource constraints, as a non-random method may not be useful enough due to the high margin of error. They offer this rule of thumb: "If no list is available and if the creation of a list is limited only by cost constraints (i.e., not by time), it would be worthwhile to sacrifice a quarter to a third of the planned sample size in order to release funds to carry out the listing." Nevertheless, random sampling is not useful when dealing with a very small sample size, since it is unlikely to be representative enough and therefore not able to provide accurate conclusions about the whole population. Purposive sampling (see Method 2) can reduce this risk. D.2 Method 2 Non-Random Sampling Purpose: To make an explicit choice based on your own judgement about exactly whom to include in your sample. When random sampling is not possible, then you can choose this sampling method for studying how primary stakeholders are affected by a project intervention. You might, alternatively, want a very specific perspective so you purposefully seek certain people or groups. As with Method 1, from an M&E perspective, the sample is needed to guide the use of information-collection methods. How to: There are two main non-random sampling methods: purposive sampling (also known as purposeful, convenience or judgmental sampling) and quota sampling.
Tips on use: Non-random sampling is more often used in small-scale monitoring or evaluation exercises and is therefore usually quicker. However, non-random sampling may not adequately represent the range of answers being sought as it involves a predetermined, and therefore potentially biased, source of information. This is because you cannot provide an estimate on sampling error. Of these options, cluster sampling can be cheaper and is easier to implement with minimal training.
This set of eight methods belong to the standard core of methods most often used for measuring changes. These methods are considered so basic to good M&E that you might well find all of them in your project’s information-gathering plan. Purpose: Stakeholder analysis in the context of M&E helps you define whom to try to involve when designing the M&E system and in which way, and it allows you to find out whose information needs must be considered. It can also be used to develop an appropriate sample for data collection (see Methods 1 and 2). This method is useful at different moments during the project:
How to:
Tips on use: Stakeholder analysis is an essential method to use in order to properly design whom to involve in which steps of the M&E process. Such a selection must be done together with different people in order to lessen the risks of having a biased selection. This is a process that continually evolves and must be repeated throughout the life of the project in order to be sure that (new) potentially important stakeholders are not missed. Purpose: To understand the historical evolution and performance of a project/organisation through its documentation, whether in written, electronic, photographic or video form. From an M&E perspective, this method can provide baseline information on a project area or a particular indicator. It also can provide a good background to activities today to help explain whether changes are occurring and why or why not. How to: 1. Make sure you are clear about the questions you wish to answer and what (type of) information you need for this. For example, "What types of income generation have been created?" 2. List all possible sources of existing information (project documentation, government records, organisation reports or geographic document records, university studies, etc.). 3. Prioritise those that are most likely to provide useful information in a cost- and time-efficient manner. This is important, especially in situations where much documentation exists. In such cases, do not try to read everything – focus on the main points. 4. Collect this documentation and check its reliability. Note contradictory evidence. Analyse it in terms of the question you were trying to answer. 5. Identify which information gaps you still have or where contradictory evidence needs to be clarified. Select another collection method, such as questionnaires (Method 8) or interviews (Method 9), to fill that gap. Tips on use: This may be a good starting point for M&E and can even serve as a substitute for the baseline (see 5.5). Such an initial review of the literature can also help you identify key issues needing to be addressed in a further M&E analysis. However, you are limited by what documentation is available and accessible, how it has been presented and by whom (possibility of biases, etc.), how it has been stored and all of the issues of quality coming from these restrictions. In this way, this method can provide an opportunity to assess an organisation’s or ministry’s internal project information collection and storage system. Method 5 Biophysical Measurements Purpose: To measure physical changes over time related to any indicator (e.g., health, nutrition, agriculture, credit) using any accepted measurement unit and procedure. From an M&E perspective, this can provide reliable, statistically verifiable data that form an important basis for measuring change and impact. How to: 1. Start by ensuring you are completely clear about what indicator or piece of data is to be monitored. 2. Agree on what the required degree of accuracy is. If a high level of scientific accuracy is needed, then expertise and an appropriate method must be sought. Counting, weighing or other measurements will depend on skills as well as type of equipment available (e.g., wooden frame, quadrats, tape measures, rulers, scales). 3. The suggested method and how it is used will then need to be adjusted to local conditions, skills and resources. Alternatively, a method can be developed together with primary stakeholders, that is mutually acceptable and is a compromise between a higher level of local appropriateness and decreased scientific accuracy. 4. The data need to be recorded in tables or diagrams, with words or numbers. These can then be used as a framework to follow in order to make comparisons over time. 5. Direct measurements can be an important part of a series of methods as described in this Annex. For example, establishing the range of impacts expected from a project with an impact flow diagram (Method 26) and then selecting a more precise measurement method to monitor a chosen impact. Tips on use: A simple measurement method that provides good estimates may well be better than a precise, more complex method that is incorrectly applied and leads to wrong data. As indicated above, the degree of accuracy very much depends on the method used. Some methods are more expensive and time-consuming than others. As direct measurement is time consuming, it is critical to be absolutely clear about how you are going to use the information before embarking on a measurement plan. Purpose: To obtain useful and timely information by observing what people do, to help make decisions on improving a project’s performance or for generating insights and findings that can serve as hypotheses for more focused studies. From an M&E perspective, this method is critical to complement collected data, can be used to understand the context in which information is collected and can help explain results. How to: 1. Agree on a clear conceptual framework, as well as guidelines for what needs to be observed and the information required. 2. Choose an appropriate observer or group of observers.
3. Collect and record data as agreed. Organise moments in which to discuss the recorded observations, not only with staff from the project and from implementing partners but perhaps also with primary stakeholders.
Tips on use: People often forget this simplest of all methods – observation. Everyone observes automatically. But you can make observation more effective by viewing it as a valid method and structuring its use. Much can be learned by watching what people actually do. Useful information and new insights can often be gained from such observation that would otherwise not be obtained. If done well, it can permit a deeper understanding of relationships within communities but also between a community and other organisations. If it is done well, direct observation can help build trust and rapport with local people and project staff. This method is also known as "participant observation" and is a common research method for social issues and processes. Direct observation is useful for validation in monitoring as it can be used to cross check responses received by other methods. There is always the danger of introducing information biases due to: biases in the observer, the way the observer influences the observed or the observed situation hampering the objectivity of the observer. These biases can never be eliminated entirely. Therefore, direct observation as a systematic M&E method should only complement other methods. Asking several people to undertake observations in the same manner can help confirm observations or identify differences and so increase the quality of the data. Method 7 Cost-Benefit Analysis (CBA)3 Purpose: To provide a format (also known as an accounting framework) to enumerate the range of benefits and costs surrounding a decision in order to help weed out costly activities that yield few benefits. From an M&E perspective, a standard use of this method is to evaluate a project by comparing actual final measurements of the costs and benefits against those proposed in the design of the project. Sometimes comparisons can be made with other projects that are delivering similar services and products. Another use is to assess costs and benefits of elements of a project, such as specific activities or indicators. How to: 1. Enlist the help of a trained economist or expert in CBA, as this method entails the use of various formulas for calculating costs and benefits as well as for discounting, marginal-return analysis and aggregation of the figures. 2. List all the project activities (potential and actual). 3. Calculate all possible project costs over the project period (e.g., labour, use of raw materials, transport). The CBA only includes costs and benefits that you define. So be clear about whether you want to include social and environmental costs and benefits. These will require more effort but will also make the CBA more comprehensive. 4. For each project activity, estimate the benefits, which may continue to occur (well) beyond the project period (e.g., 10 to 30 years). This step is more difficult than Step 3 and will require some research into and help from specific statistical formulas. Option 1. You are including aggregation in your CBA a. Aggregate project costs and benefits through the discounting formula, according to the point of interest. This is easiest through a computerised spreadsheet. b. Calculate annual net benefits by subtracting costs from benefits for each year. c. Calculate the IRR – interest received on an investment consisting of costs (negative values) and benefits (positive values) – occurring at regular periods (in this case, annually) of the series of annual net benefits. If you are using a spreadsheet programme, it will include IRR as an automatic function. d. Do a sensitivity analysis by increasing costs and/or benefits by a certain percentage (e.g., 10% or 20%) and check the impact on the IRR. If the IRR is more than the market rate of return even when costs are increased and benefits are decreased, the project is usually considered to be economically robust. Option 2. You are not including aggregation in your CBA a. Decide whether to estimate costs and benefits for individual project activities, in order to compare and choose between alternative options. b. Calculate the (potential) marginal rate of return from each activity option, by estimating the potential costs and benefits associated with alternative options for the same activity. c. Add the calculations from the previous step to the social, institutional and technical features of the activity option, to permit a more informed choice. d. Present these findings to key stakeholders for analysis and discussion, to assist in decision making. Tips on use: A CBA can be carried out at the design stage of a project to help make decisions on how the project should look and on what activities to include. The various benefits and costs can be monitored over time in order to measure changes. CBA has several advantages – but only if undertaken properly. It provides a comprehensive framework to link project costs and benefits systematically. It helps project stakeholders think about project details and gives a clear overview of how a project’s cash flows work. However, CBA is also much criticised particularly for making it difficult to account for all potential costs and benefits in a fair and equitable manner. Certain costs and benefits are very difficult to measure, such as intangible, non-financial social and environmental costs (i.e., opportunity costs). For example, how can you adequately measure the potential long-term health and environmental effects of using genetically modified seed varieties within a farming system, or the future price of tomatoes on the world market? Also, the items included in a CBA are biased according to who carries the analysis out, and therefore a CBA’s quality and coverage will vary greatly. Furthermore, there is a bias both against unknown future effects and against projects in which benefits occur later in time. Due to its complexity, CBA is usually carried out only by project designers and economists, without engaging other (primary) stakeholders. It can be made more participatory by including stakeholders in findings analysis. Different options can be presented to allow for a wider discussion on their potential costs and benefits. Participation is more obviously included in Option 1 above. 4 The mathematical complexity of the method requires CBA to be undertaken with trained economists and with appropriate computer spreadsheet programmes. Method 8 Questionnaires and Surveys Purpose: To gain data from a large number of people in a structured way according to specific questions, often in ways that allow for statistical analysis. From an M&E perspective, questionnaires and surveys form the basis of many monitoring and evaluation studies as they allow for focused data collection about specific performance questions or indicators from a sample. How to: 1. Agree on the purpose and information needs of the questions. 2. Decide whether the information needs require a questionnaire or survey format. The terms questionnaire and survey are often used interchangeably but can be distinguished as follows:
Questionnaires and surveys can range from being very simple to quite complex. These can follow a very specific and structured set of closed questions (yes/no or multiple choice questions) or they can also include open-ended questions, such as in semi-structured interviews (see Method 9). Fixed-choice or fixed-response questionnaires are good for gathering data that needs to be analysed statistically. Open-ended or free-response questions can be particularly good for determining people’s feelings and attitudes. 3. Ensure that questioning is focused and well formulated or it will not be useful (see Table D-1 below). If necessary, call in appropriate expertise to make sure that the questions have been worded correctly and can be analysed properly. The questions can be formulated to answer a hypothesis that you wish to prove or disprove (e.g., "Does a woman’s level of education affect the health of her children?") or to find out the extent of a specific problem. Table D-1. Examples of closed versus open questions5
4. Agree on who should be questioned and how many people should be included in the sample (see Methods 1 and 2). Also decide on the most appropriate manner of questioning (a form that is mailed or dropped off to be filled in independently, face-to-face individual interviews, etc.). If interviewing directly, train the interviewers so you can be sure that they understand the purpose and have the skills to ask questions in ways that limit biases. 5. Pre-test the interview questions to ensure that they are appropriate, accurate enough and give you the type of information you need. 6. Collect and analyse the information. Tips on use: Questionnaires and surveys can provide precise answers to carefully defined questions. The ease of analysing questionnaires and surveys will vary with the number of questions and the size of the sample. Often projects make the sample too big and ask too many questions. Then analysis becomes tedious, takes much time and loses its usefulness if not analysed in time for decision making. Good interviewing skills are important and come through training. If the style is very structured and inflexible, then this may inhibit openness. Long questionnaires and surveys are also tedious for the respondent. Questionnaires and surveys in which answers must fit a certain set of options or format will also fail to pick up on deviating answers and opinions. So be aware that you might be missing out on important details and variations to the questions. Questionnaires and surveys can be used with individuals or even used in group situations. However, questions in group situations may need to focus less on private issues (e.g., not on contraceptive practices or financial loans) and more on group opinions (e.g., "What are the advantages and disadvantages of different types of water wells?"). Group-based questionnaires and surveys work best in groups where members are used to working together and can trust each other, as well as the interviewer. Table D-2. Example of a mini-survey 6
Method 9 Semi-Structured Interviews Purpose: To gain information face to face from an individual or small group, using a series of broad questions to guide the conversations, but allowing for new questions to arise as a result of the discussion. From an M&E perspective, semi-structured interviews are critical for developing an in-depth understanding of qualitative issues in particular. As the interviews are open-ended (though guided by checklists), they are helpful for assessing, for example, unintended impacts (positive and negative), opinions about the relevance and quality of services and products, etc. How to: 1. Define the purpose and information needs of the inquiry and formulate an interview checklist of open-ended questions. The questions should be such that interviewees can express opinions through discussion. A logical sequence to the questions will help the discussion flow. See Table D-1 in Method 8 for ideas on how to word questions in a useful manner. 2. Agree on who should be interviewed, how many are required within the sample and whether interviews should be with individuals or in a group. 3. Gather and train a team of people to ensure that they understand the purpose and develop the proper skills (how to encourage discussion, taking accurate and useful notes, etc.). Semi-structured interviews are best conducted by two people, with one performing the interview and the other taking detailed notes. But this may not be feasible. You can try tape-recording the interviews, but this can be very inhibiting and transcribing afterwards is extremely time consuming. 4. Pre-test the interview questions to ensure that they are appropriate and accurate enough, and that the answers permit useful analysis. 5. If you conduct group interviews with more than one interviewer, it may be useful to follow the interview with a short internal discussion on the dynamics of the interview, assess the validity of the answers and decide if the interview needs to be adapted. 6. Analyse the information coming from the interviews. See Box D-7 for tips on how to structure open-ended information for easy analysis.
Tips on use: Semi-structured interviews can easily be used in combination with another method. For example, you might be walking a transect (Method 18) with farmers with whom you are having a semi-structured interview. Many of the visual group methods work best if conducted as a semi-structured interview. Semi-structured interviews can be a relaxed way to obtain insights not possible from structured questionnaires. Interesting, unforeseen topics may also emerge in this manner. However, such information may not be sufficiently precise to allow for statistically analysis. For this, use a questionnaire (see Method 8). Open-ended information is also more difficult and time-consuming to synthesise well enough to obtain clear results. It can be difficult to keep interviews focused, making different interviews difficult to compare properly. Accurate note-taking is particularly important to make interpretation possible. Take the time and money to train people to conduct a semi-structured interview. Training needs to address team preparation, interview context, sensitive listening, sensitive questioning, judging responses, recording the interview and self-critical review. 8 Group interviews require more attention to details, such as using simple language and avoiding technical jargon or expressions to be certain that the least informed person in the group understands the questions. Consider beforehand what might be politically or culturally sensitive as controversial issues can raise strong emotions leading to conflicts within the group. Purpose: To document the life story or sequence of events over time related to a person, location, household or organisation in order to obtain insight into a project’s impact – for example, how people deal with change and why change occurs in specific ways – and to learn about people’s experiences, dreams and obstacles for future planning. From an M&E perspective, case studies add life to what might otherwise be data without a human face and they allow for an in-depth understanding of the context and human factors behind general or summarised data that may be collected through other means. How to: 1. Define the purpose and precise information needs of the case study. 2. Decide how you are going to select the individuals, households or organisations about which you will do the case studies. Other methods can help with this, for example, social mapping (see Method 31 below) to find an appropriate selection of case study households. 3. Decide how you will obtain the information. If you are doing a household case study, you might wish to interview several household members and then form one study of their answers, in which you highlight similarities and differences. If you want to write a case study of an organisation, think which people you will need to interview to obtain a good overview. If you are developing a case study of a location, then you might need not only to interview people but also gather additional biophysical information. 4. Develop the question checklist that will guide the information collection. Case studies involve a written (or filmed, see Method 20 below) account of observations and answers. Attention must be paid to choosing a good interviewer/recorder, whether a person external to or within the community. In more participatory processes, the study is carried out by (a group of) individuals on themselves or each other, and perhaps with a control group for comparison. 5. Repeat the discussions with enough frequency to allow an up-to-date picture of changing conditions. The frequency will depend on the rate of change of the issues in which you are interested. Aspects that change quickly might need more frequent follow-up than issues with slow rates of change. Tips on use: The strength of this method is that you obtain much detail on a specific topic. The need for a focused case study can arise from a general survey in which a particular issue emerges as needing more in-depth elaboration. Case studies can provide interesting perspectives that you can only gain through a closer look at the overall situation (or life story) of a person, household, etc. Case studies can provide much important background and human context for data that are generated by other methods. A cross-case study analysis can be highly valuable, particularly if it relates to broader policy questions of major interest. A case study is particularly useful in complex situations where many variables interrelate and where outcomes and impact are liable to vary across different populations. However, case studies are generally not considered representative. For this reason, it is good to use case studies in combination with methods involving larger samples, such as surveys or questionnaires. A variation on this method is to use the traditional form of story-telling as an entertaining way to gain some understanding of how people deal with issues or crises. It is often an important part of village life in communicating ideas and community values. However, since a story is often a metaphor and open-ended, it needs careful thought to be useful. As with other methods, the information must be carefully recorded. Box D-8 provides one example from a booklet of case studies of various primary stakeholders participating in a project in Ghana. Offset by colourful photographs, these case studies were able to provide an attractive way of bringing human detail to an interim evaluation report.
D3 Discussion Methods for Groups Much M&E, particularly in participatory projects, can be undertaken with group-based discussions. Six basic methods to encourage discussion are presented below but there are many more. For example, many of the methods in sections D4 to D7 are also excellent for group contexts. Don’t forget the obvious discussion techniques. One of the most common ways to encourage discussion and organise ideas is with cards. You can use cards when you are brainstorming (see Method 11), working in focus groups (Method 12) or as part of other methods, such as SWOT (Method 14). After being clear about the question or topic, ask participants, either individually or in small groups of three or four, to write each idea or piece of information on one card ("one idea, one card"). The cards are pinned to the board or spread on the ground. First remove all duplicates. The group then clusters the remaining cards together into core themes. Discussion can now focus on each cluster. Purpose: To gain many ideas quickly from a group without becoming caught up in detailed discussion. It encourages people to think critically and creatively, rather than simply generating a list of options, answers or interests. From an M&E perspective, this method is often a first step in a discussion that is then followed by other methods. For example, brainstorming is useful when starting a matrix scoring exercise (see Method 32), an impact flow diagram (Method 26) or when starting to develop a stakeholder analysis (Method 3). How to: 1. Begin by asking the group to think of as many ideas as they can about the topic in question. You can give them several minutes for this. 2. Go around the group asking each person to briefly state his/her idea. The ideas can be captured using rich pictures (Method 25) or nominal group technique (Method 13), using symbols or words. Everybody’s ideas should be treated equally at this stage. Do not let people start debating each other’s ideas. 3. Once all of the ideas have been noted somewhere visible to everyone (e.g., on a flip chart or chalkboard), then there can be some analysis. 4. The emerging issues, topics and questions can later be grouped, sorted and prioritised. Tips on use: Note that this method does not, on its own, suffice as a data gathering or analysis method. The method can work with small or larger groups and can take as little as five minutes, depending on the subject, detail needed and number of people. A brainstorming session should not take very long, as it really is only meant to get out ideas that can be discussed in detail later. People find it very difficult not to comment or evaluate when ideas are generated in a brainstorm. Set a rule at the beginning that all judgements made during the brainstorm will be ruled out until a later discussion. As with most group discussion methods, some participants may dominate. To avoid this problem, you can distribute cards to all individuals on which they brainstorm their thoughts or ask them to brainstorm in sub-groups (also see nominal group technique, Method 13). This method is commonly used in combination with other methods, for example, to start a focus group session (Method 12). Purpose: To use group discussion to collect general information, clarify details or gather opinions about an issue from a small group of selected people who represent different viewpoints. It can also be used to build consensus. For M&E, focus groups are good for assessing opinions of change, assessing the quality of project services or service providers, and identifying areas of improvement. How to: 1. Determine the participants (four to eight people is ideal). Depending on your purpose, you can work with a homogenous or heterogeneous group. Alternatively, use a number of focus groups, each one fairly homogeneous, but the groups being different from each other. This enables interesting comparisons. 2. Present the group with a broad question (e.g., "What impact do you think a particular intervention has had in achieving sustainable land use?"). 3. Discuss this question for the time period agreed upon beforehand, one or two hours maximum. There should be minimal intervention by the facilitator other than to make sure that everybody has a say. Perhaps you might need to repeat the question using different words from time to time or to probe if something is not clear. 4. Take detailed notes of the discussion. Focus groups are best if facilitated in pairs - one person to facilitate the discussion and the other for note-taking. You can also record the discussion but this will have the usual problems of time-consuming transcription and inhibiting the group. 5. One way to be sure that the information collected is reliable is to keep conducting different focus group sessions until the data becomes repetitious. Tips on use: If facilitated well, this method can bring out detailed information. It generally stimulates rich responses and also provides a valuable opportunity to observe discussions and to gain insights into behaviours, attitudes, language and feelings. However, facilitation of a focus group requires considerable skill – both in moderating the group and in adequately recording the responses. Group dynamics, due to individuals being too shy, dominating, disruptive, etc. can hamper the discussion. This method can be used to obtain a consensus view. However, a small group of people cannot represent all views held by, for example, an organisation or community. On the other hand, if the group is not homogeneous enough, there can be great disagreement. So think carefully about the composition of the group. This method can generate focused insights more quickly and generally more cheaply than through a series of key informants or formal social surveys. Method 13 Nominal Group Technique (Simple Ranking) Purpose: To generate ideas and to enable a group to come to consensus in developing a ranked list of problems, issues or actions. A variant on this method is to encourage people to generate possible solutions to a given problem. From an M&E perspective, like brainstorming (Method 11), it supports other methods. It can help, for example, to generate a list of priority performance questions or indicators, to prioritise stakeholders during a stakeholder analysis (Method 3) and to follow up on impact flow diagrams to prioritise impacts (Method 26). How to: 1. Develop a list of the problems, issues or actions that need to be ranked. Express each as clearly as possible to avoid confusion. 2. Each person independently ranks the statements onto a set of cards according to his/her view of the priorities. The highest number (if there are six statements the highest number will be six) can be given to their highest ranked statement and the next highest number to their next highest priority and so on through the list. 3. The cards of rankings are then gathered and tallied on the master sheet. 4. The total scores for each statement will enable them to be put in order of importance. Variation V1. Every member of the group writes his/her ideas down as a word or phrase and then shares them one by one with the group. V2. Each idea is labelled with a symbol or letter to make ranking of solutions easier. Try to discourage discussion up until this point so that shyer members can also contribute. V3. Once all of the ideas are up on the board or flipchart, clarify, discuss and rank them as per the steps 1-4 above. Tips on use: This method can be used with small or large groups and can take from 15 minutes to an hour, depending on the size of the group and how much debate there is over the initial statements. The final outcome is a set of independent judgements made in a non-threatening, private way that will allow people to generate a group judgement without social pressure to conform. Nominal groups (groups "in name only") brings people together in a way that generates ideas more effectively and creatively than when people interact to discuss, brainstorm, and exchange information. Method 14 Strengths, Weaknesses, Opportunities and Threats (SWOT) Purpose: To identify the strengths, weaknesses, opportunities and threats in relation to a project or group, and how such an assessment will change over time. From an M&E perspective, this method is useful when qualitatively assessing, for example, the services provided by the project, relationships between project stakeholders and the organisations of the implementing partners, local groups and the project team itself. How to: 1. Referring to Box D-9, the group defines, discusses and records as many factors as possible for each heading. Emphasise that strengths and weaknesses refer to internal aspects of the group, project site or activity. Opportunities and threats can be looked at in terms of internal or external factors affecting them. 2. Alternatively, different sub-groups, for example during a workshop or in a community, can undertake a SWOT on their own. Comparing the different SWOTs can lead to a good discussion about the differences and similarities of experiences and possibilities. 3. Based on this overview, discuss what actions are needed (see Box D-10). Box D-10. SWOT window showing analysis of a mangrove reforestation project, with a resultant list of actions below9
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
