Zooming to Class?: Evidence from a Randomized Control Trial on the Effects of Online Learning on College Student Academic Achievement

How to cite

Kofoed, Michael. 2020. "Zooming to Class?: Evidence from a Randomized Control Trial on the Effects of Online Learning on College Student Academic Achievement." AEA RCT Registry. AEARCTR-0006969. https://doi.org/10.1257/rct.6969-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Michael Kofoed*

* Primary investigator

Registry ID
AEARCTR-0006969
DOI
https://doi.org/10.1257/rct.6969-1.0
First registered
Last updated
Status
Completed
Trial dates
2020-08-24 – 2020-12-11
Countries
United States of America (New York)
JEL codes
I23, I21
Registry record
http://www.socialscienceregistry.org/trials/6969

Abstract

The COVID-19 pandemic caused colleges and universities to offer many courses online. The United States Military Academy at West Point was no exception and each department was asked to find a way to teach classes at 50 percent capacity. Per institutional and Army policy, West Point already randomly assigns cadets to course order, class hour, instructor, and class section. The Economics Department decided have each instructor teach half of their course load online and the other half face to face. Given West Point's previous policy on random assignment to class section, cadets were randomly assigned to either an online or face to face class. Instructors used a uniform curriculum and all graded events were the same across the modality. This experimental setting will allow us to control for instructor style and quality while assessing the efficacy of online learning.

Experimental design

In this paper, we leveraged unique institutional characteristics at the United States Military Academy at West Point to implement a randomized control trial for a required Principles of Economics course to estimate differences in student outcomes between online or in-person instruction. In a non-COVID environment, West Point cadets (along with their peers at the Air Force and Naval Academies) have little control over their academic schedules and peer networks. For example, they do not choose which semester they are taking a specific class and are randomly assigned to academic instructors, class hours, roommates, social networks called companies, final exam periods, and military mentors.

While West Point continued to sort cadets in this matter during the COVID-19 pandemic, the Academy's administration requested each department to create a plan to use physical classroom space at fifty percent capacity. Departments were free, however, to innovate a solution to carry out this requirement. Some departments, for example, moved to a hybrid model where half of the enrolled cadets meet in person for one class meeting and the other half met online and switched for the next class time. To address this concern, the Economics Department received permission to have each instructor assigned to the introductory Principles of Economics to teach half of our sections online and the other half face to face. This course load split allows us to control for instructor quality and experience in both modalities. Principles of Economics is also a graduation requirement for all cadets, so there are no concerns about selection into the course. We randomized 550 cadets across over twelve instructors for thirty-eight class sections. In our experiment, all curriculum, graded events, homework assignments, and exams were identical across online and face to face. We also offered both online and in-person class sections for each class-hour that the course was available and we randomly assigned cadets within the hour into each modality. Cadets randomly assigned to an online class will see their grades increased by any estimated treatment effect to ensure they are not harmed by the experiment.

Design details

Randomization method
Randomization (before/during/after) COVID is done by the Office of the Dean of the Academic Board (Provost) using a computer algorithm.
Randomization unit
Classroom
Clusters
38 class sections
Observations
550 students over 12 instructors
Arms
214 in a face to face class with 337 online.
Primary outcomes
Our primary outcomes are student grades for each assignment or exam. We will also include questions on an optional course evaluation that will allow us to understand the mechanisms behind any results including questions about concentration in class, connectedness to instructor and/or peers, etc.

Cited by

No citation data available for this registration yet.

Keywords