Studying without distractions? The effect of a digital blackout on academic performance

How to cite

Garbin, Francesca; Giustinelli, Pamela. 2020. "Studying without distractions? The effect of a digital blackout on academic performance." AEA RCT Registry. AEARCTR-0006378. https://doi.org/10.1257/rct.6378-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Francesca Garbin* , Pamela Giustinelli

* Primary investigator

Registry ID
AEARCTR-0006378
DOI
https://doi.org/10.1257/rct.6378-1.1
First registered
Last updated
Status
In development
Trial dates
2020-09-07 – 2021-07-31
Countries
Private
JEL codes
I23, O33, D90, C93
Registry record
http://www.socialscienceregistry.org/trials/6378

Abstract

Rising concerns about the effects of technological distractions on concentration and learning outcomes are making us question which are the most efficient ways of studying and using smartphones. In order to investigate this issue, I assign first-year students at Bocconi University to the use of an app that helps them disconnect from distractions on their smartphones. The treatment lasts for several weeks up to the mid-term exams, and through surveys before and after the intervention I aim to detect relevant effects on academic performance, expectations about exam grades, course evaluations, and network influence.

Intervention

The objective of this experiment is to test whether using an app for blocking digital distractions is effective or not for improving academic outcomes, and eventually to quantify this effect. Students will use this app for several weeks in order to prompt possible habit changes. In case positive results are obtained, this research will be a valuable instrument to support the future implementation of a wider scheme aimed at all students.

Experimental design

I implement an experiment with first-year Bocconi students to study whether distractions coming from smartphones are detrimental to the academic performance. At the beginning of the first semester, I survey eligible students in order to gather general information about their background and their habits, and their willingness to participate in the experiment. Students are randomized into the treatment and are asked to download a blocking app on their smartphones. The app blocks other apps and their notifications (e.g. social media, messaging, news). Students are asked to activate it according to a certain schedule for the four weeks of the experiment. During the experiment, students make the conscious and intentional choice to remain off their phones by using the app.
The intervention will run in the second semester (Spring 2021) as well.

Design details

Randomization method
In the first semester randomization into treatment has not been feasible due to a small amount of participants: randomization would have prevented the eventual detection of statistically significant effects. The analysis on the first semester group ("the pilot") is carried out using propensity score matching. In the second semester students who agree to participate in the experiment will be randomly divided into two groups. The control and treatment groups will be randomized according to some stratifying variables, possibly subject to changes depending on the number of respondents. Randomization will be done using the statistical software Stata.
Randomization unit
Randomization is conducted at the individual level.
Clusters
The goal is to have at least several hundred students involved, in order to have hundreds of treated individuals. A maximum of 400 individuals can be treated, as this number corresponds to the number of app licenses currently purchased. In particular, in the second semester a maximum of 310 licenses will be made available.
Observations
The design is not clustered, so the number of clusters and the number of observations correspond.
Arms
The goal is to have an almost balanced number of treated and control students.
Primary outcomes
The analysis will use as outcome variables: grades and GPA both in the first semester and in the following ones; students' expected and perceived performance before and after taking the exams; students’ appreciation of courses; realized study time, that might be affected by the treatment; other academic expectations. Heterogeneous effects may be assessed thanks to app data on frequency and length of breaks during the digital blackouts, and thanks to network structure position. Treatment effects may be analysed in the form of intention-to-treat analysis.
Secondary outcomes
The analysis may be focused on other relevant factors in the education production function and on time usage. Survey questions collect a wide variety of both pre-intervention and post-intervention information that can be exploited in the analysis. Family inputs can be analysed in order to detect heterogeneous patterns. Elements related to socio-economic background, well-being, personality traits, study habits, COVID-19 risk perception, technology use and history, academic expectations, and social networks may be used as well.

Cited by

No citation data available for this registration yet.

Keywords