![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Evaluating Clean Energy Programs
More Information
For information about direct technical assistance on energy efficiency program EM&V, contact Niko Dietsch (dietsch.nikolaas @epa.gov).
- Types of EM&V
- Basic Approach to Measuring Impacts
- Planning and Conducting an Impact Evaluation
- Two Approaches to Energy Efficiency Impact Evaluation
- Additional Resources
Evaluation, measurement, and verification (EM&V) encompass a wide range of analyses used to assess clean energy programs. The goals of EM&V include:
- Determining whether overall objectives are being achieved
- Identifying any necessary program improvements
- Assessing program cost-effectiveness
- Estimating impacts and their persistence over time
- Capturing energy (kWh) and demand (kW) impacts in energy planning
EM&V efforts are applied to energy efficiency programs administered by utilities, states, and third-party energy efficiency providers , as well as state-mandated renewable energy policies and programs funded by ratepayers.
Types of EM&V
Three types of EM&V, each with a different purpose, are frequently undertaken to assess program and policy effectiveness.
| Evaluation Types | Purpose | Information Derived |
|---|---|---|
| Impact Evaluation | Quantifies the direct and indirect benefits of a program or project using measured or deemed savings methods (described below). | Determines the quantity of energy and/or demand saved, the monetary value of these savings; can include the amount of emissions reductions and other non-energy benefits. |
| Process Evaluation | Indicates how to improve the structure and delivery of a program or project. These evaluations typically survey program stakeholders, analyze their feedback, and use this information to identify opportunities for program improvement. | Determines how well program or project processes are performing and provides recommendations for how these systems they can be improved. |
| Market Effects Evaluation | Indicates how a program affects the structure or functioning of a market – or the behavior of participants in a market – that result from one or more program efforts. | Determines changes that have occurred in state operations and/or private markets, and evaluates how the marketplace is different as a result of the program. |
This Web page focuses on impact evaluations, which estimate the key direct benefits of clean energy policies and programs. Additional resources can further assist with each of the evaluation types.
Basic Approach to Measuring Impacts
Estimating the energy , demand , and environmental impacts from clean energy policies and programs is a critical evaluation step. How this is done depends on whether the effort being evaluated aims to increase clean energy supply or reduce energy demand (i.e., energy efficiency).
With clean energy supply programs, a primary metric of interest is energy output, which can be directly measured using standardized industry equipment. When this clean energy output displaces electricity generated from utility fossil fuel generation or the direct combustion of fossil fuels, the quantity of energy displaced can be used to evaluate the resulting environmental benefits.
With energy efficiency programs, the key metric of interest is energy savings. This quantity cannot be directly measured. Instead, efficiency program impacts are estimated by taking the difference between:
(a) actual energy consumption after efficiency measures are installed
(b) what energy consumption would have occurred during the same period had the efficiency measures not been installed (i.e., the baseline)
In addition, steps can be taken to adjust the baseline and/or the post-installation energy use to account for factors other than the energy efficient measure or system that affect energy consumption (e.g., weather, building occupancy, operating hours). Energy savings are therefore determined using the equation:
Energy savings = (b)(Baseline energy use) – (a)(Post-installation energy use) ± (c)(Adjustments)
In the graph below, energy savings are represented by the shaded area.
Source: National Action Plan for Energy Efficiency (2007). Model Energy Efficiency Program Impact Evaluation Guide (PDF). (152 pp, 1.4 M, About PDF)
Planning and Conducting an Impact Evaluation
Conducting a clean energy impact evaluation involves the following steps:
- Set the evaluation objectives in the context of the program and policy goals.
- Select an evaluation approach (see below) and prepare a program evaluation plan that takes into account critical evaluation issues.
- Implement the evaluation and determine program impacts, such as energy and demand savings and avoided emissions.
- Report the evaluation results and work with stakeholders to implement program improvements, as appropriate.
- Incorporate results into future program and resource planning activities.
Adopting a formal evaluation planning process allows for participants to clearly and transparently communicate objectives and approaches, and increases the likelihood that the information will be applied in a timely manner to achieve overall program goals.
More information on planning for and conducting a program impact evaluation is provided in the National Action Plan for Energy Efficiency's Model Energy Efficiency Program Impact Evaluation Guide (PDF). (152 pp, 1.4M)
Two Approaches to Energy Efficiency Impact Evaluation
A key step in planning a program impact evaluation is to select an approach to estimating energy and demand savings. The two key options are a deemed savings approach and a measured savings approach.
For programs targeting simpler efficiency measures with well-known and consistent performance characteristics, a deemed savings approach may be appropriate. This method involves multiplying the number of installed measures by an estimated (or deemed) savings per measure, which is derived from historical evaluations.
Deemed savings approaches may be complemented by on-site inspections.
For larger and more complex program strategies – including those expected to result in significant savings or those with a high degree of uncertainty – a measured savings approach that follows established protocols may be appropriate. Estimates of energy (and/or demand) savings are calculated using one or more of the following techniques.
| Measured Savings Approaches to Impact Evaluation | |
|---|---|
| Engineering Methods | Standard formulas and assumptions are used to calculate the energy use of the baseline and post-installation energy systems. |
| Statistical Analyses | Statistical models are used to estimate “before” and “after” scenarios, while taking into consideration changes in weather, facility occupancy, factory operating hours, and other factors that affect energy use. |
| Computer Simulation of System Performance | Computer models are used to predict the change in energy use after complex, system-wide improvements in energy efficiency are implemented . These models are typically calibrated with actual performance data. |
| Metering and Monitoring | Baseline and post-installation energy use is directly metered and monitored, while accounting for the non-energy factors that affect energy consumption. |
| Integrative Methods | Integrative methods combine some or all of the preceding approaches. For example, metering and engineering methods can calibrate computer simulations of baseline and post-installation buildings that receive efficiency retrofits. |
States that apply measured savings approaches typically adopt an evaluation protocol to guide their efforts, increase transparency, and identify the appropriate level of rigor. For these reasons, such a protocol can be particularly important when impact evaluation results are used as the basis for decision making. The International Performance Measurement and Verification Protocol (IPMVP) (PDF) (93 pp, 2.5M) 
is one commonly-used example.
Regardless of the approach selected, a key goal for impact evaluation is to minimize uncertainty while balancing evaluation costs with the value of the information received.
Additional Resources
The Nation Action Plan for Energy Efficiency (Action Plan) Model Energy Efficiency Program Impact Evaluation Guide (PDF) (152 pp, 1.4M) provides:
- A framework that jurisdictions and organizations can use to define their “institution-specific” evaluation requirements.
- Standard evaluation planning and implementation process that can be used for calculating savings.
- Definitions, best practices on key evaluation issues, and a list of evaluation resources.
The Action Plan offers direct technical assistance to state public utility commissions on evaluation planning, as well as an upcoming series of conference calls on key evaluation topics. Contact Niko Dietsch (dietsch.nikolaas@epa.gov) for more information.
The CALMAC Web site provides a comprehensive resource for program impact evaluation guidance. The site contains:
- California Energy Efficiency Evaluation Protocols: Technical, Methodological and Reporting Requirements for Evaluation Professionals (PDF). (292 pp, 1M)
- A searchable database of over 400 evaluation reports on a variety of general and program specific topics.
The Consortium for Energy Efficiency (CEE) Market Assessment and Program Evaluation Clearinghouse provides a source of evaluation reports for programs throughout the country.
The Regional Evaluation, Measurement and Verification Forum is facilitated by the Northeast Energy Efficiency Partnerships (NEEP) to develop consistent EM&V protocols for energy efficiency and other demand-side resources in the Northeast and mid-Atlantic regions, with the objective of supporting priority energy and environmental policies.
The U.S. EPA's State Technical Forum explores key analytical questions and issues surrounding state clean energy efforts. Forum resources for evaluation, measurement, and verification (EM&V) include background information, stakeholder presentations, and links to additional information.