The Impact of Online Lectures on Student Learning

How to cite

Zölitz, Ulf; Shan, Xiaoyue. 2020. "The Impact of Online Lectures on Student Learning." AEA RCT Registry. AEARCTR-0006538. https://doi.org/10.1257/rct.6538-1.2

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ulf Zölitz* , Xiaoyue Shan

* Primary investigator

Registry ID
AEARCTR-0006538
DOI
https://doi.org/10.1257/rct.6538-1.2
First registered
Last updated
Status
Completed
Trial dates
2020-09-14 – 2021-04-01
Countries
Switzerland ()
Registry record
http://www.socialscienceregistry.org/trials/6538

Abstract

Digitization of higher education is fundamentally changing students’ academic experience and their learning process. Traditional live lectures are increasingly being replaced with online lectures. We conduct a field experiment at a major research university where students are randomly assigned to live lectures versus watching these exact same lectures online. We estimate whether assignment to online lecture affects students’ exam performance and academic experience.

Intervention

In the exceptional circumstances prompted by COVID-19, universities around the world have massively increased the number of courses taught online. Even before 2020, the digitization of higher education began to transform the academic experience and learning process for university students. However, very limited research exists on how students are impacted by the shift to online lectures.
In this paper, we investigate how the introduction of online lectures affects student performance and university experience. We study a European research university that was forced to introduce online lectures as part of social distancing measures in 2020. In this setting, we assign students to a rotating attendance schedule that determines whether students attend live lectures or watch the same lectures online. Assignment to live or online lectures is determined by a random number – the last digit of the student identification number.
We aim to answer the following research questions:

1. Do students perform better on exam questions when the content of the question was covered in an online lecture compared to a traditional in-person lecture? (Exam performance)
2. Do online lectures affect subsequent attendance, course dropout, course passing, and student grades? (Academic trajectory)
3. Do online lectures affect students’ social interactions, satisfaction with different course features, study habits, and overall learning experience? (Subjective outcomes)
4. Does assignment to online attendance affect study dropout, completion of the first year, elective course choices, and major switching? (Longer run outcomes)

Experimental design

Our field experiment takes place at a European research university. All first-year bachelor and master students are randomly assigned into five attendance groups. The assignment is based on a random number – the last digit of the student ID number. Based on a rotating block schedule, each attendance group is permitted to attend some lectures in person, and others online via Zoom. More specifically, each attendance group is allowed to come to the university two days per week . The randomization into attendance groups implies that, for any given lecture, some students receive the content in person and others receive it via an online lecture. Our within-subject design means that a given student attends some lectures online and others in person. Therefore, our experimental design creates exogenous variation in whether a given lecture was attended online and in the share of lectures a student attended online.
By linking lecture content to exam questions, our design allows us to compare student performance based on whether they attended a lecture online or in person. In other words, we will evaluate the impact of online lecture assignment on student performance through exam questions covering material taught in the corresponding lecture.

Design details

Randomization method
All first-year students are randomly assigned into five attendance groups based on the last digit of their student ID number. Each week, each group can come to university for two full days, according to a rotating block schedule. This randomization implies that for any given lecture, some students receive the content in person and others receive it via an online lecture.
Randomization unit
Randomization into live and online lectures takes place within students. A given student will attend some lectures live and some online.
Clusters
About 1000 students.
Observations
We expect to observe at least 1,000 students. For each of these students we expect to observe at least 50 responses to different multiple choice exam questions resulting in an overall sample of about 50,000 observations.
Arms
We expect a sample size of 50,000 exam questions. So about 10,000 questions in the control group and 40,000 questions in the treated (online) group.
Primary outcomes
- Exam question answered correctly - Subsequent lecture attendance - Course dropout - Course passing - Student grades
Secondary outcomes
- Student subjective outcomes - Study dropout - Completion of first year - Elective course choices - Major switching

Cited by

No citation data available for this registration yet.

Keywords