Improving Online Learning through Course Design: A Microeconomic Approach
How to cite
Shaikh, Hammad. 2022. "Improving Online Learning through Course Design: A Microeconomic Approach." AEA RCT Registry. AEARCTR-0008787. https://doi.org/10.1257/rct.8787-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Online education has expanded dramatically over the past two decades, yet significant learning challenges remain. In light of these, my paper provides the first microeconomic analysis to examine how the quality of online university courses can be enhanced through course design, addressing the twin needs of providing individualized support to students and keeping them engaged with online coursework.
First, I gather rich data on undergraduates at a large public university taking an online introductory programming course that has a cumulative structure. The data allow me to monitor students' study time precisely and to characterize important dimensions of heterogeneity: student attentiveness and whether they are forward-looking. I then conduct two randomized interventions that nudge students to utilize an online discussion board more fully and to complete online assignments. I then develop and estimate a behavioural model of student effort supply, credibly identifying the marginal benefits and costs of effort at each stage of the cumulative learning process using the two field experiments. The estimated model allows me to explore the efficacy of changing assignment grading weights to improve student learning. In contrast to the actual (equally-weighted) grading scheme, simulated weights that maximize learning are decreasing across assignments, serving to increase effort by myopic students early in the course when they acquire foundational skills. My course-design approach is applicable more generally in other online and traditional course settings.
Intervention
The interventions considered in this study can be categorized as `targeted informational reminders' as their design includes the following elements: 1) they prompt students to take a specific action (e.g., registering for the discussion board), 2) they provide information on how to clearly execute the action (e.g., instructions to sign-up for discussion board), and 3) they serve as a reminder for the specified task (e.g., course uses a discussion board).
Intervention 1: discussion board sign-up activity
The sign-up activity is designed to promote discussion board registration. The activity is composed of the following elements: 1) it presents a link to the discussion board sign up page, 2) it uses screen shots to illustrate key steps for sign up, 3) it summarizes all steps into an animated GIF, and 4) students are disclosed information about the discussion board. The online activity consists of two pages: the first page contains the instructions, and the second includes information about the discussion board. The informational page discusses the functionality of the discussion board, and also discloses the proportion of students' questions that have been answered by either a peer or the instructor. Students in the treatment group are randomly assigned to receive either sign-up instructions only (i.e., page 1), discussion board information (i.e., page 2), or both.
Intervention 2: homework reminder message
The reminder messages are aimed at promoting students to further participate in their weekly low-stakes homework. Reminders are only sent for the graded homework assessments after week 2. The homework reminder is composed of the following three elements: 1) reminding students of the upcoming homework deadline, 2) prompting them to set aside time in their schedule to next make progress on the homework, and 3) including a direct link to the homework assessment.
Experimental design
Intervention 1: discussion board sign-up activity
For all the students who completed the baseline survey and did not register for the discussion board within the first week of the course, half of them received the sign-up activity.
Intervention 2:
For students who had not completed the homework before the deployment of the reminder message, half of them are randomly assigned to receive a homework reminder. The reminder messages were sent throughout the course and were re-randomized with each deployment. Consequently, the number of total homework reminders a student receives follows a binomial distribution with 10 trials and a 0.5 probability of success.
Design details
Cited by
No citation data available for this registration yet.