Decomposing the Wedge: Mechanisms driving the gap between projected and realized returns from energy efficiency investments

How to cite

Christensen, Peter; Francisco, Paul; Myers, Erica. 2018. "Decomposing the Wedge: Mechanisms driving the gap between projected and realized returns from energy efficiency investments." AEA RCT Registry. AEARCTR-0002985. https://doi.org/10.1257/rct.2985-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Peter Christensen* , Paul Francisco , Erica Myers

* Primary investigator

Registry ID
AEARCTR-0002985
DOI
https://doi.org/10.1257/rct.2985-1.0
First registered
Last updated
Status
On going
Trial dates
2017-08-01 – 2018-10-01
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/2985

Abstract

We will evaluate the effects of performance-based bonus pay on the quality of work done by contractors to improve building shell tightness in the Weatherization Assistance Program, the largest residential energy efficiency program in the United States. In almost all WAP jobs, the measures contractors are asked to perform are chosen based on engineering calculations and quality control is done through an inspection process. Because they are not compensated directly on the energy performance of the home, contractors lack an incentive to seek out and execute any additional cost effective measures that they find on site, such as air sealing. We will evaluate whether performance-based bonus pay can result in cost effective improvements in the quality and quantity of air-sealing performed by contractors. We will also examine the effects of performance incentives on work done by bonus-eligible contractors that is excluded from the bonus payment.

Intervention

In this study we will evaluate the impact of financial incentives to contractors on air sealing results and overall energy efficiency improvements. This will be conducted in the state of Illinois due to strong connections with the state weatherization program. In Illinois, a formula is used that estimates the amount of air sealing expected based on a pre-retrofit air leakage test; the leakier the home, the greater the expected air leakage reductions. The housing sample will be jobs that are done outside of Chicago, which represents about half of the state weatherization program. Chicago-based jobs will be excluded because the agency that runs the Chicago program already compensates their contractors for air sealing based on measured airtightness outcomes, whereas the rest of the state compensates contractors using pre-set sums that are based on the degree of expected air leakage reductions. In this study some of these non-Chicago homes will be eligible for an additional incentive to contractors based on air leakage reduction (the “treatment” groups) and others will not (the “control” group which will only be eligible for the pre-set compensation).

Our intervention consists of bonus payments that are made to contractors who achieve CFM50 readings below the minimum required by the state, which is 10% above the “target” that is based on the starting value. (The 10% allowance by the state takes into account that some homes have characteristics that make achieving target unusually difficult.) All contractors are made aware of the bonus program through a series of program meetings preceding the program year and the magnitude of incentive payments are clearly printed on the work order that each contractor receives at the outset of a job. Our intervention consists of two different bonus regimes: a “high” payment ($1.00) and a “low” payment ($.40) per CFM50 reduction reductions beyond 10% above target. The control jobs will not be eligible for bonus payments.

The selection of air sealing as the basis for our study stems from its importance in delivered energy savings. Air sealing is deemed to be cost-effective in all homes, with the assumption that air sealing is permanent and therefore provides savings throughout the life of the home. As such, every home receives air sealing. Additionally, air leakage sites can vary widely from house to house, and so there is more potential for the attention and skill of the contractor to have an impact on identifying and properly sealing the leaks. Third, the metric for assessing airtightness (a blower door test) has a finer resolution than metrics for other common measures such as insulation, providing an opportunity to compensate based on a wide range of possible outcomes.

The blower door test assesses the volumetric flow rate of air from the home when the home is depressurized by a set amount (50 Pascals [Pa]). The leakier the home, the more air needs to be moved to cause that amount of depressurization. The unit of measurement from the blower door test is cubic feet per minute at 50 Pa (CFM50). It is standard practice to conduct blower door measurements before and after a home is weatherized. These measurements are already required for WAP homes. Lower CFM50 values indicate that the home is better air-sealed, and thus well suited to retain heat.

Experimental design

Our experiment offers bonus payments to contractors who achieve CFM50 readings below the minimum required by the state, which is 10% above the “target” that is based on the starting value. (The 10% allowance by the state takes into account that some homes have characteristics that make achieving target unusually difficult.) We will have two different treatment arms, one with a “high” payment ($1.00) and one with a “low” payment ($.40) per CFM50 reduction reductions beyond 10% above target. The control jobs will not be eligible for bonus payments.

In addition to the data from WeatherWorks and utilities, we will also be tracking information directly from agencies on the inspector reports. Each month, a representative from each weatherization agency will be asked to report on the number of contractor call-backs, and what the call-backs were for. We will be merging the information with data in WeatherWorks based on the job number.

Design details

Randomization method
When a new job is initiated in the WeatherWorks system, it will automatically be assigned to the “high” payment treatment, “low” payment treatment, or control. The overall aim of the project is to have group populations of 25%, 25%, 50%, respectively. Randomization is done through custom software built into the program that generates the work orders. The state of Illinois expects to weatherize approximately 1500 homes outside of Cook County in program year 2018. 1200 of these will be done on single family homes for which we have utility data. Therefore, we expected roughly 300 jobs for each of our 2 treatments and 600 for the control group. If a job is assigned to one of the two treatment groups, there will be a message printed on the front of the work order indicating that it is eligible for a $1.00 or $.40/ CFM50 bonus, how that bonus will be calculated, and how to receive payment.
Randomization unit
individual job
Clusters
1200 jobs
Observations
1200 jobs
Arms
300 “high” payment treatment, 300 “low” payment treatment, 600 control
Primary outcomes
CFM50 reductions, Reductions in energy consumption as assessed on utility bills, Quantity and type of findings and deficiencies tabulated from post-weatherization inspection reports
Secondary outcomes
CFM50 reductions at concurrent but ineligible jobs, CFM50 reductions as a function of time after intervention (learning)

Cited by

No citation data available for this registration yet.

Keywords