Monitoring, Evaluation, and Learning (MEL) Plan
In each of the previous steps, you have based decisions on the available evidence and identified how to manage risk before implementing your selected strategies by addressing any information needs. Now you may be wondering how you will generate the evidence during implementation to understand if your strategies are on track to achieving your needed outcomes toward your desired goals, and when needed, to understand why or why not they are on track. This is where monitoring, evaluation, and learning (MEL) comes in.
Monitoring is the periodic collection and analysis of data related to goals, objectives, and/or key variables that may influence expected results. Monitoring enables you to generate the data necessary to evaluate the effectiveness or your strategies towards achieving goals, learn, and improve their effectiveness as needed. Some also refer to this process as MEL. MEL design is the approach taken to structure MEL, including sampling methods, use (or not) of controls and comparisons, timing of interventions, and timing of observations.
MEL should be primarily for your team’s benefit so that team members know if your strategies are on track and what adjustments may be needed to achieve desired conservation outcomes. MEL helps your team determine what is working and what is not working, and when needed, understand why. This, in turn, allows your team to adapt as needed and improve its effectiveness.
Some misconceptions about MEL:
#1. MEL is the domain of scientists or professionals with advanced graduate degrees. In reality, most MEL efforts are something that project staff can and should do. One reason this guide spends a lot of upfront time on defining the context and design of your action plan is because investing time on these initial steps will help make developing and implementing the MEL plan straightforward for those involved in the process. You may find that specialized expertise is needed or having an external expert will reduce potential bias in addressing some information needs.
#2. The first step in developing a MEL plan is to ask, “What data should I collect?” Although this is an important question, it is usually premature to ask without having the means to narrow down the answer. Fortunately, you have completed Step 1 and parts of Step 2 of the process, including identifying indicators for your goals and objectives and identifying other information needs in each step to help you manage risk. This work will help you define the rest of your data collection to address other information needs.
#3. Monitoring and evaluation require complex methods and specialized skills. Methods do not necessarily need to be complex. In fact, gathering the necessary information using a simple method when possible is preferable. While the data you gather in some cases may be less reliable, it may be sufficient to manage risks associated with implementing activities.
#4. Monitoring and evaluation is too costly to do. Following from above, if you focus your efforts on only the most important information needs and keep your methods as simple as possible, monitoring does not have to be costly. As a rule of thumb, monitoring comprises around 10% of a project budget. However, if after using available evidence there are still information needs and more extensive data collection and analysis needed to manage risk, then the team should plan for dedicating more time and resources to monitoring.
In this section you will learn how to use your planning work to develop a monitoring plan. A monitoring plan is important because it provides the blueprint for how monitoring will happen and succinctly organizes and summarizes a lot of information.
How To
Developing a MEL plan involves four major steps:
- Define your audiences and their information needs, starting with the team’s (for whom).
- Identify and define your data needs (what).
- Determine your methods for collecting data (how).
- Specify responsibilities and timeframes (when, where, and who).
The following sections will guide you in completing each of these steps.
Define and prioritize your audiences and their general information needs
This step involves broadly identifying your audiences and their information needs. Once you have completed this step, you can start thinking about MEL efforts to address information needs.
You can begin by listing the audiences that will need information from your project. The first audience on your list should be the project team itself. Many times, team members think of MEL as a requirement for satisfying external demands for accountability. While this may be part of the reason behind MEL, it should not be your only or even your primary reason. Ideally, MEL will primarily serve the information needs of the project team to improve effectiveness. Systematic MEL can provide project teams with valuable information about how to improve their conservation actions. If we want our conservation efforts to be effective, we need to learn from our experiences and integrate those lessons into current and future projects.
Next, you should consider other audiences outside of your team, such as project partners (who are not part of the project team), donors, local residents, policymakers, other conservation organizations, the broader conservation community, academics and students, and the general public. Your audience will also include several, if not all, of the actors or groups of actors identified in your stakeholder analysis.
Some examples of audiences and information needs:
- Project team: Understand if implementation of actions and achievement of objectives and goals are on track, why or why not, and what adjustments may be needed to improve effectiveness.
- Project funders: Accountability for performance in implementation and outcomes.
- Your organization: The contribution to conservation for gaining buy-in and support for its mission.
- Conservation community: Lessons on the effectiveness of actions towards outcomes and under what conditions for future programming decisions.
You can capture this information in a simple table like Table 1, which is based off of our Marine Reserve example.
Identify and prioritize the specific information needs for each of your audiences
In order to begin thinking about what you will be monitoring for your project, you should be clear about what specific information you should generate and share with each of your key audiences to address their needs within what timeframe. To do this, you should determine what specific information needs will be used for each audience and for what decision-making purpose.
For each audience in the table from the previous step, write a statement or statements that describe specifically what your audience (including the team itself) needs to know to make a decision and within what timeframe.
It is likely that you will identify more information needs for audiences than you have time and resources to address. In addition, it is unlikely that all information needs are equally important to address and you will need to prioritize. Efficient monitoring uses the minimum amount of financial and human resources to provide the information at the right time to address specific needs of key audiences for decision-making.
Define the data needed to address each information need
Keep in mind that data gathered to address one information need could address the information needs of several different key audiences. You should maximize such opportunities for more efficient data collection. What may vary though is how you present information to your audience.
In addition to tracking progress towards achievement of goals and objectives, other information needs may focus on the status of factors that you are not actively addressing but that you should track to better understand why or why not you are making progress and if a new action will be required. For example, you may also want to monitor the external context of your project including key contextual factors you have identified (e.g., changes in climate, spread of invasive species, political factors). This may help you determine if – even though the project is well implemented and has a sound theory of change – factors originally thought to be outside the current manageable interest of your project may be influencing the effectiveness of your actions. Similarly, you may want to track the population of a non-target species to see if it is stable, in which case no action will be required, or declining, in which case you may have to incorporate an action.
You may also decide to monitor outcomes that are of special interest to stakeholders. In our Marine example, the donor is interested in how the project is progressing (Table 1), but may specifically want to know about the profitability of the new fishing techniques being promoted. As such, the team should consider collecting data on this intermediate result in their results chain, even though they did not develop an objective for this result.
Determine your MEL design
MEL design is the specific approach used to collect data. Box 1 details the criteria for a good MEL design. In developing a MEL design, you should aim for the most cost-effective approach that will give you data that are reliable enough to address the information need of the audience. Often teams will want to use the most accurate and reliable design, but you may be able to get data that are sufficiently accurate and reliable for you to make management decisions using a feasible and cost-effective option.
For example, if you needed to monitor how much monkey meat was sold in local markets, your methods could include:
- Weighing an average monkey and multiplying the number of monkeys sold by this average weight,
- Using a produce scale to weigh all monkey meat sold to the closest kilogram and summing these values, or
- Using a chemist scale to weigh all monkey meat sold to 5 decimal places of accuracy and summing these values.
All of these methods may provide the information needed to make a decision about adapting your strategy, but each method varies in its level of effort, cost, reliability, and accuracy. The first option may be the most efficient use of resources for the accuracy and reliability needed to inform the management decision.
The ideal suite of data collection methods for your project is not always feasible to implement. Strive for what you and your team can realistically achieve given your resources and time.
Selecting methods involves 4 main steps:
1) Determine whether you can get data from secondary sources.
You should first determine if the data you need are available from existing, reliable sources. Assuming the methods used by secondary sources meet the criteria for good methods, you should try to use these data rather than spending your project resources on collecting primary data. In some cases, you may not be able to get exactly what you need from secondary sources, but you should evaluate whether the data you can get would be sufficient to address the information needs. If so, you could consider modifying your indicator so that you can draw from this existing source. You should be careful, however, that your new indicator does in fact truly serve as a good measure of your information needs. One potential advantage to using outside data sources is that your external audiences may view the data as more neutral and, therefore, more credible.
Good sources of data include ongoing research projects and routine monitoring by scientific institutes, universities, or administrative bodies. For example, one method for collecting data about a given fish population might be to download harvest records posted by a government agency on the Internet.
2) If you cannot get data from an existing source, research available methods to generate data.
There may be a wide range of possible methods to generate data for a given indicator. In many cases, you or your colleagues will know the range of methods available. If this is not the case, you can learn about various methods by talking to experienced people, reviewing documents or manuals on the subject, taking courses, or scanning through examples of monitoring plans that have been developed by other teams working on similar projects.
3. Apply criteria for selecting the most appropriate method.
In choosing your monitoring methods, you should review the criteria for good MEL design (see Box 1). If you are choosing among more than one method for a given indicator, you should choose the method that best meets all the criteria. This is not always a simple task and will require that your team give careful thought to the different methods available and the importance of the different criteria for your project.
The proposed method should be referenced or summarized in a few words in the monitoring plan. If the method is not well known to those carrying out the monitoring, it may be necessary to define and describe the method more fully in a separate document.
Looking at our marine example again, potential methods for the chosen indicators might be:
- Check the registry of fishing gear on boats before they leave on fishing expeditions
- Visit the boats during fishing hours to make sure they are using only the alternative fishing techniques
If you review each of the methods above, you will see that they meet the criteria for good methods, although typically, there will be some trade-offs in terms of how well they meet each criteria. For example, it may be cost-effective to check the registry of fishing gear on boats involved in the project before they leave on fishing expeditions, but the accuracy of this method might not be as high as hiring someone to visit the boats during fishing hours to make sure they are using only the alternative fishing techniques. The latter option, however, would be much more expensive. In this case, the team also chose to do random checks, which are less expensive but might help them determine if their first indicator is suitable. In choosing a method, your project team needs to consider what is acceptable for each criterion you consider.
4) Determine whether you need an additional method.
As with indicators, you may determine that you want to measure something using more than one method. This may be because you are not fully confident that any of the methods available to you are accurate, but if two or more methods give you similar results, your confidence in the accuracy of the results will be increased. In evaluation terminology, this is known as methods triangulation. The random checks of fishing boats discussed above are an example of this tactic. As another example, you may be interested in knowing how much timber is being harvested from a forest. You could check the records of timber companies or the government forestry agency, but you are not sure how accurate these will be. So, you might also estimate how many logs fit on a truck and then calculate the average number of trucks that leave the area per week. This will help you determine how reliable your methods are. If they consistently give you the same information, you might consider eventually eliminating the more difficult or costly method.
Specify responsibilities and timeframes (where and who)
In addition to your indicators and methods, your monitoring plan should include other information that is important for those implementing it. At a minimum, it should include information about where the data will come from (i.e., the data source), when it will be collected, and who will be collecting it. Your monitoring methods, indicators, and related goals, objectives, or results should be captured in a table like the one shown in the Example section below (Table 2). The remainder of this section describes the minimum amount of information that would be included in a monitoring plan table.
1) Specify the timeframe and frequency of data collection.
You should define the dates when baseline and final data will be collected for each indicator. In many cases you will want to collect data more frequently than this (e.g. quarterly or annually throughout the duration of your project). In deciding when and how often you should collect data, consider the following factors:
- Time period to effect change. If you realistically cannot expect to see a change in a factor for five years after the start of the project, then your next measurement after the baseline measurement should probably be no earlier than five years (unless you need to monitor it for the influence of other variables).
- Natural variability of the phenomenon to be monitored. If what you are monitoring varies naturally, you should have enough data points taken at appropriate time frames so that your data are not influenced by natural variations that have nothing to do with project-related impacts. For example, if you are collecting data influenced by climatic changes, you should clearly note if the measurement time coincides with an El Niño year and how that might affect your results. You may also want to vary the number of collection times around the El Niño event to compensate for this effect.
- Seasonality issues in terms of data availability and variation. You may need to always monitor at the same time of year, or alternatively, at various points of the year to be able to factor in seasonal changes. For example, if you are monitoring water levels, they will vary widely depending upon whether you take them at the beginning of the wet season versus during or at the end of the wet season. In most cases, it would not make sense to compare water levels taken at the end of the dry season one year with those taken at the end of the wet season the following year.
- Project life cycle. This is a more practical concern. You should keep in mind if you have key project reviews, planning, reports, or other project-related events on the horizon. Adjust your monitoring times to meet those needs if it will not substantially affect the outcome of your monitoring.
2) Specify individuals responsible for data collection and analysis.
Monitoring can require extensive resources, especially commitments of project team members’ time. It is important to ensure that the appropriate person(s) with the right skills are designated to handle these functions. While multiple staff may be responsible for collecting, recording, and analyzing data, it is often important to have a single driving force and “owner” of the overall monitoring process. You should state the name of the individual or the organization responsible for measuring each indicator and the name of the person in the project team responsible for getting the information (when this is not the same person).
Again, the details for how your monitoring will happen should be recorded in table format. You can use Table 2 below as a template for your project’s monitoring plan.
3) Specify how information will be used for decision making and how information will be shared with the broader conservation community.
The ability to make sound decisions about investing limited conservation funds requires the conservation community to strengthen the evidence underpinning our assumptions.Through sharing your team’s lessons learned (e.g. Shared Projects in Miradi Share), over time, with concerted effort, the conservation community can contribute to and improve the evidence base and the effectiveness and efficiency of conservation actions. The conservation community is more prone to share information about achieving outcomes and less often about the times when they are not achieved. We encourage teams to share lessons about the challenges that they encountered in achieving desired results in their context.
Example
The following is an example of a monitoring plan, based on the Marine Reserve example. Note that, in some cases, the team chose to use more than one indicator or more than one method to make sure that they were adequately measuring the variables of interest.
Exercise
Part 1: Define audience and information needs for monitoring
For your project, define your audience and information needs by following these steps:
- Make a list of your audiences, starting with your project team
- Identify other potential audiences
- Identify the general information needs for each of your key audiences
Record your analysis of audiences and information needs in a table like the one shown in Table 1.
Part 2: Define indicators and monitoring methods for a goal and an objective
Refer to your audience and information needs table and results chain to help you identify where you need to develop indicators – especially those related to the goals and objectives you have defined. Choose at least one goal and one objective. For each, define at least one indicator using the following steps:
- Use the text of your SMART goal or objective to define your indicators
- Review your criteria for a good indicator and make sure your indicators comply
- Modify your draft indicator as needed to make sure it complies with the criteria
- Determine whether you need an additional indicator
- For each goal and objective, also develop methods
- Determine whether you can collect data from existing sources of information; if you cannot collect your data from an existing source, research methods available
- Apply criteria for selecting the most appropriate method
- Determine whether you need an additional method
- Record your indicators and methods in your monitoring plan using the format in Table 2 to organize your information
Congratulations!! You have now finished Step 2 of the Conservation Standards.
Conservationtraining.org students can now click here to complete the Step 2 Quiz.