The Importance of Evaluating Energy Efficiency Financing Programs

by Chris Kramer, Energy Futures Group

As financing continues to develop into an important strategy in the pursuit of energy efficiency objectives, it is becoming ever more important to understand what this strategy can likely achieve. Several states have expressed public aspirations to shift away from the use of ratepayer-funded energy efficiency incentives and toward leveraging of private capital as a primary strategy for achieving energy efficiency goals. Many energy efficiency advocates, however, have expressed concerns that such a shift could lead to a loss of efficiency savings, outweighing any reductions in program costs. Proponents on each side of this discussion have pointed to examples to support their positions, yet few formal impact evaluations have been conducted to shed light on financing’s incremental contribution to efficiency efforts, much less its potential to replace other core programs.

Some of the risks related to a potential loss of energy efficiency may not be directly observable without employing established evaluation methods to determine how much financing has actually “grown the pie.” For example, even with high participation rates, it may not be obvious which customers would have moved forward using other private financing in the absence of program offerings. Similarly, it may not be readily apparent whether the efficiency level of installed measures is incrementally higher than customers would have otherwise chosen. Given the imperative to increase savings above and beyond existing levels to meet efficiency and climate goals, these questions are not trivial. Evaluation, measurement, and verification (EM&V) methods have been developed over the past several decades to address these key issues, such as free ridership and savings relative to baseline. To date, however, formal EM&V techniques have seldom been applied to financing.

This situation is beginning to change. California, for example, has recently hired a team of evaluators to wrestle with these questions in connection with the launch of a new suite of energy efficiency financing pilots. Massachusetts is in the midst of conducting an evaluation that will compare the costs and benefits of its rapidly expanding HEAT Loan program. New York has also developed an evaluation plan and set of metrics for its new Green Bank, setting aside up to $4 million to conduct evaluation activities. In several cases, regulatory involvement has been important to ensure that financing programs are held to the same standards of accountability and transparency as other traditional efficiency programs.

Essential to these efforts, particularly in those jurisdictions that are considering a shift toward financing and a corresponding reduction in incentives, is the issue of “additionality.” That is, what level of savings can financing programs produce on top of investments that occur “naturally” in the marketplace? Answering this question requires an analysis of savings levels that can be directly attributed to financing, particularly if the intent is to reduce other ratepayer-supported strategies as financing expands. Critically, early thinking within the evaluation community suggests that this type of analysis may be possible with minimal burden on capital providers (who may not be accustomed to efficiency program reporting requirements) because (1) the basic data they need from capital providers may be collected through the regular course of program implementation, and (2) most of the analysis would focus on customers rather than lenders. Ultimately, the key question is how financing changes customer behavior in favor of greater efficiency—a question that can only be answered by carefully exploring customer choices.

One other early revelation is that cost-effectiveness ratios, often produced by evaluation studies, may not provide a full picture of financing’s impact. For example, from a utility standpoint, shifting toward financing may reduce the level of program expenditures and improve the ratio of benefits to costs, even as a drop in total participation could also reduce the sum of net benefits achieved. Although the cost-effectiveness ratio may improve within the program, less overall efficiency could mean greater supply-side investment is needed to make up for the loss. Viewed in totality, the end result of this two-sided equation could actually be an increase in the overall cost of energy supply. As a result, evaluations conducted with an eye toward financing’s potential to replace other strategies may need to focus as much on total net benefits produced as on the cost-effectiveness ratio achieved.

These critical issues will be essential to work through as energy efficiency programs turn increasingly toward financing to help achieve big-picture goals. For these reasons, policymakers and regulators may wish to consider requiring formal evaluations of financing programs within their jurisdictions, alongside ongoing evaluations of other traditional programs. Ultimately, EM&V should help inform all stakeholders about both the benefits¬ and the challenges of financing strategies, leading to more prudent programmatic choices in the future.