An Experimental Analysis of Adoption, Savings and Information Diffusion for Industrial Energy Management Systems

How to cite

Wolfram, Catherine; Greenstone, Michael; Knittel, Christopher. 2018. "An Experimental Analysis of Adoption, Savings and Information Diffusion for Industrial Energy Management Systems." AEA RCT Registry. AEARCTR-0003227. https://doi.org/10.1257/rct.3227-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Catherine Wolfram* , Michael Greenstone , Christopher Knittel

* Primary investigator

Registry ID
AEARCTR-0003227
DOI
https://doi.org/10.1257/rct.3227-1.0
First registered
Last updated
Status
On going
Trial dates
2016-09-16 – 2020-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/3227

Abstract

This study will examine whether there is under-investment in diffusion of new technologies due to information spillovers. We will explore potential market failure attributable to within-industry spillovers of unilateral marketing and education activities. If technology suppliers cannot internalize all of the producer surplus from marketing and education, government subsidization of these activities could lead to more efficient diffusion rates.

In particular, we will analyze information spillovers for Lightapp’s industrial energy efficiency technology for air compressors. We will first randomize marketing and subsidies for the technology across industrial facilities in Pacific Gas and Electric and Southern California Edison service territories. We will then use a stepped wedge randomized design to estimate the software’s energy and bill impacts. After removing references to Lightapp, we will share these results with a random selection of previously uncontacted firms. We will elicit willingness to pay for the Lightapp software from these firms and from a set of control firms that receive no information. This will enable us to compare the effects of education about one energy efficiency product on willingness to pay for a similar energy efficiency product, potentially offered by a competitor.

Experimental design

We are using a stepped-wedge design with randomized recruitment and treatment. Details are below.

Step 1: Sampling and recruitment

Excluding the pilot sites, the sampling frame consists of 1,124 facilities. After establishing this sampling frame, we randomly assigned facilities into two groups:
ˆ FREE group: 70 percent of facilities were o ered free Lightapp services
ˆ DISCOUNT group: 30 percent of facilities were o ered a 25 percent discount for Lightapp services
For both FREE and DISCOUNT groups, we fully subsidize the one-time installation of Lightapp
software devices and any subsequent maintenance or repair costs incurred during the duration of the
project.
We also cross-randomized the timing in which Lightapp's sales team will contact each facility to o ffer
its services. We randomly sorted the facilities in our sampling frame into an ordered list, and divided
facilities into 50 sequential tranches according to this randomized ordering. Each week, we released an
additional tranche of facilities to Lightapp's sales team. Information included company names, addresses,
and contact information. The release date initiated a rolling 30- or 40-day recruitment window, during
which Lightapp was encouraged to contact potential program participants and convert them into Lightapp
customers. After this tranche-speci fic recruitment window closed, Lightapp agreed not to recruit any
facility in the tranche that had not signed a contract for the remainder of the experiment.
We made three adjustments to the experimental design near the beginning of the recruitment phase.
We expanded the recruitment window from 30 days to 40 days after discovering that this change would
give more facilities sufficient time to gain internal approval and sign contracts with Lightapp. We also
increased the tranche size from 12 to 14 facilities due to the unexpectedly large number of facilities
Lightapp deemed ineligible. We maintained the same randomization order for determining the facilities in
each tranche. At the same time, we removed water and sewage treatment facilities from the experiment.
These facilities initially comprised approximately 20 percent of our sampling frame, yet they were very unlikely to have compressed air onsite. When we removed water
and sewage treatment facilities, we preserved the randomization order among all other facilities. We
had released eight tranches of facility names to Lightapp when we introduced these three experimental
design changes. We will perform sensitivity analysis on key results excluding facilities recruited prior to
instituting these changes.
Following the approach by Hussey and Hughes (2006) for power calculations in stepped wedge designs,
we pre-specifi ed a target sample size of 100 Lightapp participants. Due to funding constraints, we made
this target a ceiling, so recruitment would end after 100 facilities signed participation contracts. Since
Lightapp successfully recruited 100 customers before we released all facilities in the sampling frame, we
hit this ceiling and stopped recruitment. At this time, there were 117 uncontacted facilities remaining.

Step 2: Installation and Validation
After the end of the recruitment window, Lightapp and the other installers have 80 days to install the
EMS software in the new participating facilities. This process requires connecting the Lightapp system
to air compressors. A group of installation subcontractors perform these installations. If the lead installer, Ingersoll
Rand, deems an installation more expensive than authorized in their contract, Lightapp or another
subcontractor, Compressor IQ, performs the installation. At the time of registering this pre-analysis
plan, two individuals from Ingersoll Rand had overseen every installation, ensuring that installations
were performed correctly and consistently.
After installation is complete, Lightapp validates the data by comparing readings from the newly
installed meters to the PG&E or SCE electricity meter readings. Lightapp explores and addresses any
data discrepancies.

Step 3: Baseline data collection
Following meter installation and validation, each facility in the project enters into a baseline data
collection period of no less than 90 days. During this time, no one at the participating facility has
access to the Lightapp software platform or receives any performance reports or alerts. This establishes
a "business as usual" compressed air energy usage baseline for each facility. Note that we will test for
the presence of Hawthorne eff ects.

To ensure that the facilities begin the treatment phase (Step 4) in the established randomization
order, we occasionally extend baseline periods for some facilities. If one facility experiences a delay in
treatment period commencement, Lightapp withholds data from all facilities later in the randomization
order. We plan to look for the existence of heterogeneous impacts with respect
to baseline period length.

Step 4: Treatment period data collection
After this baseline period is over, facilities begin receiving real-time Lightapp data. This treatment
period lasts 12 months. During this time, facility personnel have access to all EMS data, scheduled reports,
and noti cations with tips and recommended actions for improving compressor eciency. Noti cations
include performance summaries and alerts for energy use data anomalies that cannot be explained by
changes in production. Participants can also compare their Compressed Air System Efficiency (CAS)
scores, which reflect the energy required to produce 100 cubic feet per minute (CFM), to other sites in
the project. All data are anonymized, but users can sort based on the general industry. Companies with
multiple plants that have Lightapp access can directly compare detailed data across these sites. We limit
the number of facilities that can begin the treatment period to three facilities per week.

Design details

Randomization method
Random number generator on a computer.
Randomization unit
Establishment.
Clusters
NA
Observations
100 establishments
Arms
We randomized the timing of treatment, as described above.
Primary outcomes
Take-up of free and subsidized energy management system, energy savings from energy management system, information spillovers to subsequent customers.

Cited by

No citation data available for this registration yet.

Keywords