Experimenting with Labs

How to cite

Handler, Nils; Martin, Ralf. 2022. "Experimenting with Labs." AEA RCT Registry. AEARCTR-0009105. https://doi.org/10.1257/rct.9105-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Nils Handler* , Ralf Martin

* Primary investigator

Registry ID
AEARCTR-0009105
DOI
https://doi.org/10.1257/rct.9105-1.0
First registered
Last updated
Status
In development
Trial dates
2022-09-30 – 2022-12-31
Countries
Private
JEL codes
Q51, Q52, Q53
Secondary IDs
/
Registry record
http://www.socialscienceregistry.org/trials/9105

Abstract

The Laboratory Efficiency Assessment Framework (LEAF) is an environmental accreditation scheme designed to reduce the environmental impact of university research labs. It is key to achieve Imperial College London’s target of becoming a sustainable net-zero carbon institution by 2040. This paper analyses its effect on saved costs, including energy, and carbon emissions leveraging a randomized controlled trial (RCT), the gold standard methodology on the Maryland Scale of impact evaluation. By randomizing the LEAF programme roll-out to a select number of laboratory buildings, we isolate its causal treatment effect from potential confounders. Further, we analyze whether a top down vis-a-vis a bottom-up implementation is more effective in increasing uptake. By providing rigorous evidence on the program’s average treatment effect of 3,000 pounds total cost and 8,5 tons of CO2 savings per lab space, this study seeks to inform potential replications in other higher education institutions in Great Britain and beyond.

Intervention

The Laboratory Efficiency Assessment Framework (LEAF) is an environmental accreditation scheme designed to reduce the environmental impact of university research labs. It is key to achieve Imperial College London’s target of becoming a sustainable net-zero carbon institution by 2040. This paper analyses its effect on saved costs, including energy, and carbon emissions leveraging a randomized controlled trial (RCT), the gold standard methodology on the Maryland Scale of impact evaluation. By randomizing the LEAF programme roll-out to a select number of laboratory buildings, we isolate its causal treatment effect from potential confounders. Further, we analyze whether a top down vis-a-vis a bottom-up implementation is more effective in increasing uptake. By providing rigorous evidence on the program’s average treatment effect of 3,000 pounds total cost and 8,5 tons of CO2 savings per lab space, this study seeks to inform potential replications in other higher education institutions in Great Britain and beyond.

Experimental design

We randomize at the buildings level in order to minimize potential spillovers. Thus the buildings can be considered as randomization clusters.

Sample Size: As you can see from the randomization table in the annex, there are 23 buildings, which comprise a total of 1460 laboratory spaces associated with them.

We use the STATA randtreat-command to carry out the randomization into three treatment arms of equal size, namely two treated and one control.

Design details

Randomization method
We re-randomized increasing the integer seed until we obtained a randomization satisfying two conditions: (1) to be balanced, and (2) to not include any department into both of the two treated arms to implement clean treatments. (1) Balance is verified with the balancing table with no stars. This indicates no significant difference between any of the treatments, hence balance. Primarily, the number of lab spaces is broadly comparable across the three treatment arms. (2) The second condition results from our experimental design, specifically the exogenously varied rollout either via top-down instruction from the heads of department or via a bottom-up approach where laboratories can individually sign up themselves. Therefore, departments must not form part in both top-down and bottom-up treatments. We obtained this desirable result setting the seed at seven, after iteratively increasing the integer value. The second condition precluded us from applying more elaborate approaches such as the minimum-maximum t-statistic out of 1000 draws described for example in Bruhn $&$ McKenzy (2009). We randomly assign the two misfits globally, that is among all three treatment arms.
Randomization unit
We randomize at the buildings level in order to minimize potential spillovers. Thus the buildings can be considered as randomization clusters.
Clusters
23
Observations
Roughly 1500 lab spaces in 23 buildings
Arms
LEAF programme rollout: Bottom-Up 8 buildings, 417 laboratory spaces Control 8 buildings, 476 laboratory spaces Top-Down 7 buildings, 587 laboratory spaces
Primary outcomes
The data collected using qualtrics measures a number of variables which are aggregated to the two primary outcome variables, namely total cost and carbon savings per lab. The variables include figures such as the number of waste bags collected per week and their average weight, the number of fume cupboards, their hours of daily operation and number of days used during the week, their face velocity, aperture and percentage time that the sash opens; the number of safety cabinets, their power demands measured in watts, their average hourly usage per day and week, and whether they are ducted; the number of PC monitors, their level of brightness and time until sleep mode sets in; the number of cold storage units, their age, their operating temperature, the average ambient temperature, whether their filters are clean and how often their doors are opened, etc. Further, as no two labs are the same and the type of equipment varies between labs and natural science disciplines, the categorization also includes an additional bespoke category where other potentially very energy-intensive laboratory equipment can be added, such as ovens or lasers for example. The primary outcomes are calculated as difference between endline and baseline values for each building.
Secondary outcomes
A secondary outcome variable that we measure is the electricity consumption at the building level. To do so, we clean the raw electricity consumption data compiled by the Sigma-software. We reduce its frequency from half-hourly measurements each day to monthly data. We compute compound annual growth rates from month to month. In the light of error-prone measurements of faulty transformer-meters, we filter out values that exceed growth rates of 100 percent from one data-point to the next. Further, we aggregate the values from individual transformers to their respective buildings by reconstructing a system-inherent algorithm that assigns individual grid-transformers to buildings within Imperial College.

Cited by

No citation data available for this registration yet.

Keywords