Testing and Screening Energy Efficiency Programs

by Robin LeBaron and Kara Saul-Rinaldi

Cost-effectiveness testing is crucial to the future of energy efficiency programs. Commissions and utilities use cost-effectiveness tests to make decisions about which energy efficiency and other demand-side programs to fund. If a new energy efficiency program doesn’t pass the tests, it may never be created. And in some states, programs are being constrained or even eliminated as a result of these tests.

If the cost-effectiveness tests were always accurate, this wouldn’t be a problem. But in many places, the difficulties involved in designing a good test are preventing programs that are in the public interest from going forward.

The National Home Performance Council (NHPC) is proposing a solution in the form of a new framework for the tests. The Resource Value Framework (RVF) proposes a number of strategies for ensuring that the tests provide decision makers with accurate information. They include a worksheet to promote transparency and balance in test inputs, alignment of the tests with existing state policies, and use of best practices.

Energy Efficiency by the Numbers

Cost-effectiveness testing is a method for determining whether energy efficiency, demand response, fuel switching, and other demand-side programs make economic sense. This raises the question of economic sense for whom? To address this issue, in the early 1980s the California Energy Commission and the California Public Utilities Commission developed five tests, each
designed to ask this question from a different perspective.

The Utility Cost Test, for example, compares the financial savings that the utility realizes as a result of the reduced demand with the utility’s (or program administrator’s) costs to run the program. If a whole-house retrofit program costs the utility three cents per kilowatt-hour saved, and the utility spends eight cents per kilowatt-hour to purchase electricity, the program would pass this test.

The Societal Cost Test (SCT) and the Total Resource Cost Test (TRC) take a much broader viewpoint, trying, in general terms, to compare all costs and all benefits. (Despite its name, the TRC considers fewer costs and benefits than the SCT.) Benefits include the expenses that the utility does not have to incur to purchase energy and capacity resources, as well as potential transmission and distribution and other savings that result from the energy efficiency (or other demand-side) program. The costs include both the expense of running the program and the cost to the participant of making the improvement.

In theory, the SCT and the TRC are good indicators of the value of a program. In the absence of one of these tests, a utility might develop a program that looks beneficial to both the participants and the utility, but would not really be as cost-effective as the least expensive way to meet the energy need in question.

In practice, however, the SCT and TRC are challenging to conduct well because so many of the benefits are hard to quantify. In the case of whole-house energy efficiency programs, for example, participants consistently say that the increased comfort resulting from reduced air infiltration and better distribution of conditioned air is one of the most important reasons they upgrade their home. But most commissions and utilities do not include comfort and similar “nonenergy” benefits in TRC tests because they are notoriously hard to measure, even though these factors can be much more significant for many homeowners than energy savings.

Other often-neglected but real benefits include reductions in water use and savings in other fuels, including natural gas (when an electric utility is sponsoring the program), oil, or even wood.
There are other problems with the tests as currently implemented. Because the California Standard Practice Manual provides very little practical guidance regarding the nuts and bolts of conducting the tests, each state has developed its own approach to discounting rates, calculating avoided costs, addressing the challenges of participant benefits, and so on. Although there is room for variation, some approaches are arguably better than others for arriving at a result that accurately reflects reality.

And then there is the question of policy. Many states have adopted policies to guide energy policy. Some, for example, seek to assist low-income households with energy costs. These goals are not always taken into account when testing.

These problems are not academic. The TRC is used as the primary test in 34 states, and the SCT is used in several others. Utilities, commissions, and other decision makers rely on these tests to decide whether to launch and fund programs. If the tests were perfect, this would be appropriate. But the tests often do not provide accurate information, preventing commissioners and other decision makers from having the resources necessary to make determinations about funding for energy efficiency programs.

Solutions: Best Practices and a New Resource Value Framework

The RVF is a set of guidelines and tools for designing and implementing tests so that they provide commissioners with the information necessary to make decisions about demand-side programs. The RVF recommends that the following principles be taken into account in the design and implementation of cost-effectiveness tests:

  • Recognition that the overarching goal of cost-effectiveness testing is to determine whether a program is in the public interest;
  • Alignment of the tests with state energy policy goals, which are an indication of the public interest in that particular state;
  • Symmetry between costs and benefits, such that if a type of cost is included in the test, the corresponding benefits should also be included;
  • Consideration of relevant benefits, even if they are difficult to quantify, through use of one of a number of methodologies discussed in the paper;
  • Transparency through documentation of assumptions and methods.

To provide a guide to realizing these principles, the NHPC is developing a spreadsheet that can serve as a tool for documenting test inputs and assumptions, and ensuring that any given test is aligned with energy policy goals and is symmetrical with respect to costs and benefits.

The RVF also recommends use of a set of “best practices” in the design and implementation of cost-effectiveness tests, including full accounting for appropriate avoided costs and use of a discount that reflects the low risk of energy efficiency as a resource.

The RVF can be used to structure and ensure that the principal screening tests in the California Standard Practice Manual (the SCT, the TRC, and the Utility Cost Test) are designed according to best practices. The framework can also be used by states to develop a rigorous test that meets their specific policy goals. In the spring of 2014, the NHPC will launch a new Energy Efficiency Screening Coalition to encourage dialogue and public understanding of this new approach.

Energy efficiency has the potential to be an important resource for meeting states’ energy needs. To fully realize this potential, we need tests that provide the right information. The NHPC’s current efforts are designed to help us reach that goal.