A randomized controlled experiment for comparing the effectiveness of hybrid and online teaching during COVID-19 pandemic

How to cite

Cheung, Yue Yat Harrison; Lam, Kwok Fai. 2021. "A randomized controlled experiment for comparing the effectiveness of hybrid and online teaching during COVID-19 pandemic." AEA RCT Registry. AEARCTR-0008750. https://doi.org/10.1257/rct.8750-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Yue Yat Harrison Cheung* , Kwok Fai Lam

* Primary investigator

Registry ID
AEARCTR-0008750
DOI
https://doi.org/10.1257/rct.8750-1.0
First registered
Last updated
Status
Completed
Trial dates
2020-09-01 – 2020-12-23
Countries
Hong Kong ()
Registry record
http://www.socialscienceregistry.org/trials/8750

Abstract

This study aimed to compare the effectiveness of hybrid and online teaching through a large-scale randomized controlled experiment. A total of 725 students were randomly assigned to either the hybrid or online teaching mode where students assigned to the hybrid mode received face-to-face lectures and tutorials and those to the online mode received exactly the same live-streamed lessons on Zoom. Since all students had access to the recorded teaching videos, the face-to-face element was the only difference between the hybrid and online teaching modes. This experiment can help to identify the add-on value of face-to-face teaching in a hybrid mode over a purely online teaching mode. Results showed that the difference in learning outcomes between the two modes were statistically insignificant. Class size was found to be an effect modifier that students assigned to the hybrid mode had significantly higher final weighted and final exam scores if they had face-to-face lessons with 25 students or less. The Pass/Fail grading option was found to have significantly negative effect on course performance. Continuous assessment should be emphasized and contribute more toward the final grade so that students are encouraged to work hard throughout the entire semester.

Intervention

Students in each course were randomly assigned to the intervention and control groups where students in the intervention group received hybrid mode instructions and those in the control group received purely online mode instructions.

Experimental design

A total of 725 undergraduate and postgraduate students from the University of Hong Kong taking one of the 7 statistics courses were invited to participate in this study. Students in each course were randomly assigned to the intervention and control groups where students in the intervention group received hybrid mode instructions and those in the control group received purely online mode instructions. The assignment score, test score, coursework score (a weighted average of the test and assignment scores), final exam score and final weighted score (a weighted average of the final exam and coursework scores) along with the baseline variables were collected for the purpose of this study.

Design details

Randomization method
Stratified randomization was done in office by a computer. The strata were the 7 statistics courses. Randomization was conducted within each stratum, i.e., each of the seven statistics courses that participated in the study. For the two large courses with enrollment over 140 (i.e., STAT1600 and STAT1603), students were randomly assigned to either the hybrid (i.e., intervention) or online (i.e., control) teaching mode in ratios of 1:2 and 1:3, respectively, whereas students in other smaller courses were randomly assigned to the two modes in a ratio of 1:1. The difference in the assignment ratio was to comply with the COVID-19 measure that the number of students in a classroom could not exceed one third of its maximum capacity.
Randomization unit
Individual students.
Clusters
Randomization units were not clusters. However, there were seven strata, i.e., the seven statistics courses that participated in the study.
Observations
725 students.
Arms
282 students were assigned to the intervention group, while 443 students were assigned to the control group.
Primary outcomes
The assignment score, class test score, coursework score (a weighted average of the class test and assignment scores), final exam score and final weighted score (a weighted average of the final exam and coursework scores) were collected for the purpose of this study.

Cited by

No citation data available for this registration yet.

Keywords