Understanding the Importance of Summer Learning in a Pandemic

How to cite

Brough, Rebecca; Corcoran, Sean; Gibbs, Chloe; Sullivan, James; Kroeger, Sarah. 2020. "Understanding the Importance of Summer Learning in a Pandemic." AEA RCT Registry. AEARCTR-0006392. https://doi.org/10.1257/rct.6392-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Rebecca Brough* , Sean Corcoran , Chloe Gibbs , James Sullivan , Sarah Kroeger

* Primary investigator

Registry ID
AEARCTR-0006392
DOI
https://doi.org/10.1257/rct.6392-2.0
First registered
Last updated
Status
On going
Trial dates
2020-05-18 – 2023-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/6392

Abstract

This study seeks to better understand the role of individualized, virtual summer instruction in redressing learning losses resulting from the COVID-19 school closures. In particular, we are interested in the potential for virtual learning to combat the negative effects of time out of school on achievement gaps. Given the mixed evidence on virtual learning platforms, especially for disadvantaged populations, the prospect of repeated school closures in the upcoming school year points to an urgent need to identify optimal strategies for online education. Our research design is based on a randomized-control trial evaluation of a unique summer program in NYC which will be offered free of charge to public school students in grades 3-8. All study students will have access to the virtual study materials, and treatment group students will additionally receive synchronous virtual instruction and comprehensive supports. Online assessments will be conducted in math and reading for all students in the study, providing not only a measure of program impact but also a baseline description of students’ skill levels following a shortened school year.

Intervention

We are working with Practice Makes Perfect (PMP), an organization in New York City that is about to launch a guided online summer learning program for up to 5,000 mostly low-income public school students. This 8-week program will consist of synchronous, interactive classes in math and reading, including one-on-one time between students and teachers, and will be offered at no cost to the students. Given the extraordinary academic circumstances students faced this past spring, we expect there will be great interest in this summer program. This program will be evaluated using a randomized controlled trial. Eligible and interested students will be randomly assigned to one of two treatment arms. The first arm will receive the full program, and the second arm will receive a light-touch version of the program: access to the same set of online materials used by the treatment group, but without a dedicated instructor or the format of the guided lessons. All fieldwork will be conducted remotely, including recruitment, online learning, surveys, and skill assessments.

Experimental design

The research team is interested in understanding how best to offer education services to students via. a virtual setting. The COVID crisis has demonstrated a need for society to understand how best to offer educational services virtually. The summer months already pose a unique equity challenge as students of low-socio-economic backgrounds are at greater risk of experiencing learning loss, as compared to other groups. This, in combination with the varied type of academic support students across NYC received in the spring semester of 2020, demonstrate a need to understand how best to administer summer learning programs.

This study will operate as a randomized controlled trial. Students interested in participating in the summer program will be randomly assigned to one of two groups. The treatment group will receive access to online platforms as well as to mentors and teachers through virtual platforms. The control group will have access to online platforms, but not to live-virtual sessions with teachers or mentors.

Design details

Randomization method
This study will use STATA programming to randomize students. Randomization will be stratified by students grade and zip code. Prior to randomization, ordered lists were created for each zip-code grade pair. Each ordered list was split into blocks of 6. Each block of 6 was then randomly ordered using a randomly generated number between 0 and 1. The 3 lowest-ranking values of each 6-block group was assigned to the treatment group, the remaining 3 were assigned to the control group. On June 22nd, new lists were created to reflect a 70-30 Treatment-Control ratio.
Randomization unit
Randomization occurs at the student level. Randomization is stratified by student grade and zip-code and randomized in blocks of 6 (ensuring that 3/6 students were assigned to the treatment group). Starting June 22, students were randomized using a 70-30 ratio (70% assigned to treatment group; 30% assigned to control).
Clusters
NA
Observations
Approximately 11,000 students.
Arms
We have successfully enrolled 10,839 students into the study with 6303 in the treatment group and 4536 in the control group.
Primary outcomes
Our main outcome of interest is student academic progress over the 8-week course. All study participants will complete online assessments in math and reading at the beginning and end of the program. While we also plan to collect MAP (Measures of Academic Progress) scores from administrative records, the ongoing crisis may prevent schools from conducting these tests in Fall 2020. In light of this risk, a high completion rate of the skills assessments by students in both experimental arms is critical. We plan to use both monetary incentives and non-monetary nudges to encourage completion of the assessments. Non- monetary incentives will include verbal confirmation and congratulations from the instructor, and visual cues to track completion within the virtual program.

Cited by

No citation data available for this registration yet.

Keywords