Using Online and Text Message Coaching to Improve Student Outcomes During the COVID-19 Pandemic

How to cite

Petronijevic, Uros; Logel, Christine; Oreopoulos, Philip. 2021. "Using Online and Text Message Coaching to Improve Student Outcomes During the COVID-19 Pandemic." AEA RCT Registry. AEARCTR-0007000. https://doi.org/10.1257/rct.7000-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Uros Petronijevic* , Christine Logel , Philip Oreopoulos

* Primary investigator

Registry ID
AEARCTR-0007000
DOI
https://doi.org/10.1257/rct.7000-1.0
First registered
Last updated
Status
Completed
Trial dates
2020-05-11 – 2020-09-01
Countries
Canada (Toronto, Ontario)
Registry record
http://www.socialscienceregistry.org/trials/7000

Abstract

Many college students struggle with mental health concerns, such as anxiety or a lack of belonging, and are often unaware of on-campus support services. The recent, rapid shift from in-person to online learning caused by the COVID-19 pandemic potentially exacerbates these issues. Focus-group evidence in the spring of 2020 from over 500 students at a large Canadian university shows that many students experienced loneliness, lacked structure, and felt unmotivated to continue coursework online when COVID-19 lockdown measures began. Motivated by these findings, we then designed an online module and text-messaging platform to help students navigate these problems, experimentally evaluating the effects of the program on mental health, well-being, and GPA in a sample of over 1,600 students enrolled in courses at two large Canadian universities during the summer of 2020. In addition to course grades, outcomes include self-reported measures of overall physical and mental health.

Intervention

The intervention takes place at two Canadian universities among all students taking first-year economics courses during the summer semester of 2020. After completing a short warm-up survey, students are randomized into either a control group that receives a few additional survey questions or a treatment group that is asked to read stories of recent student experiences in isolation and how they overcame challenges and regained a positive sense of control. At the end of the survey, treated students are invited to provide their cell phone number to receive weekly messages on study tips and mental health advice. All students are then surveyed again at the end of the semester in a follow-up survey to gather data on mental health and well-being.

Experimental design

Students enrolled in 2020 summer semester courses at two large Canadian universities go online to complete a warmup exercise. After completing a short warm-up survey and declaring whether they give consent for their data to be used for anonymous research purposes, students are randomized into either a control group that receives a few additional survey questions or a treatment group that is asked to read stories of recent student experiences in isolation and how they overcame challenges and regained a positive sense of control. At the end of the survey, treated students are invited to provide their cell phone number to receive weekly messages of study tips and mental health advice. Students are then surveyed again (using the same online platform) at end of the course to gather measures of mental health and well-being, as well as academic outcomes.

All students in first-year economics courses during the summer semester of 2020 at the two universities are required to complete the online exercises in order to receive 2 percent of their course grade. The course grade is awarded only for completion of the exercises, regardless of whether students provide consent for the research team to use their data.

Subgroup analysis: Treatment effects on all outcome variables mentioned above will be explored in the main sample and across four subgroups:

- Students with high versus low initial locus of control.
- Students initially struggling with life during the COVID-19 pandemic.
- Students experiencing financial insecurity initially.
- International students.

Our focus-group evidence, gathered in the spring of 2020 (prior to the experiment) at University 1, showed that students who took control of their new situations during the pandemic lockdowns by establishing new routines and a sense of purpose coped much better overall. We therefore gathered baseline measures of students' Locus of Control with the aim of exploring heterogeneous treatment effects across students who feel more or less control over their lives. This is our primary subgroup of interest.

Secondary subgroup analysis involves the following groups:

- Students initially struggling with life during the COVID-19 pandemic.
- Students experiencing financial insecurity initially.
- International students.

We gathered data on these variables during the initial survey and will explore whether treatment was more effective among students initially struggling with life under the pandemic, experiencing financial insecurity during the pandemic, and among international students who potentially face different implications from the pandemic owing to travel restrictions and time-zone differences.

Design details

Randomization method
Randomization is done using random digits in student numbers.
Randomization unit
Randomization is at the student level
Clusters
none, although our sample is drawn from two different Canadian universities.
Observations
1700 students in total 900 at one university; 800 at other university
Arms
The sample at each university is equally (and randomly) divided between treatment and control groups. The approximate breakdown is the following: University 1, Control group students: 450 University 1, Treatment group students: 450 University 2, Control group students: 400 University 2, Treatment group students: 400
Primary outcomes
Mental health and well-being outcomes gathered from the follow-up survey at end of summer semester.
Secondary outcomes
GPA and credits earned from summer semester

Cited by

No citation data available for this registration yet.

Keywords