The Effect of Priming on Retention and Learning in a MOOC

How to cite

Shotland, Marc; Siddiqui, Hira; Naimpally, Rohit. 2015. "The Effect of Priming on Retention and Learning in a MOOC." AEA RCT Registry. AEARCTR-0000679. https://doi.org/10.1257/rct.679-4.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Marc Shotland* , Hira Siddiqui , Rohit Naimpally

* Primary investigator

Registry ID
AEARCTR-0000679
DOI
https://doi.org/10.1257/rct.679-4.0
First registered
Last updated
Status
Completed
Trial dates
2015-03-30 – 2015-12-31
Countries
United States of America (Global)
JEL codes
I21
Registry record
http://www.socialscienceregistry.org/trials/679

Abstract

Massive Open Online Courses (MOOCs) are characterized by relatively lower retention and pass rates than equivalent in-person courses. One type of dropouts (Type 1) is made up of enrollees for whom the course may not be appropriate. For these students, the optimal set-up might be one that allows students to “shop”—enroll in a course, discover whether the content or level is appropriate, and then drop out if need be after feeling sufficiently informed, but before too much time has been invested in making the choice. Another type of dropouts (Type 2) may be made up of enrollees for whom the course is appropriate, and who intend to complete it, but face small barriers that inhibit them from doing so. Discouragement could be one of those barriers. If learning objectives are imprecise, students may underestimate the relevance of the course, or focus on the wrong teaching points. This could lead to poor performance, which in turn feeds discouragement and ultimately fuels drop out. Therefore a model that makes learning objectives clear upfront could lead to reduced attrition over time.

Our research proposes to identify a simple intervention that allows potential dropouts (type 1 and type 2) to self-select into either type as soon as possible. In addition, it would seek to help students who self-select into the second type (i.e. those who want to complete the course ex-ante) by reducing some of the barriers that they may face, ultimately leading to higher course completion rates. The intervention we propose includes giving students a Baseline Diagnostic Test at the start of the course. This diagnostic may help type 1 students understand exactly what the course covers, and what measurable outcomes they are expected to learn, and whether or not the content is right for them. It may also help spark early engagement with type 2 students, giving them a preview of the concepts covered in the course, allowing them to focus on the right teaching points, and motivating them to complete the course. Using our online course JPAL101x: Evaluating Social Programs, we aim to run a randomized evaluation of this intervention.

Intervention

Intervention 1: Baseline Diagnostic Quiz with answers
The intervention will test whether taking a “Baseline Diagnostic Quiz” yields better retention and performance in the course.

The Baseline Diagnostic Quiz is placed in the first unit of the course. It is ungraded and responses are optional. It asks technical questions about the different concepts that are covered throughout the course. In this intervention, the correct answers to the questions will be shown to the students, allowing them to assess their baseline understanding of the concepts covered in the course.

Intervention 2: Baseline Diagnostic Quiz without answers
The intervention will test whether early exposure to key course concepts yields better retention and performance. Furthermore, will leaving the diagnostic questions unanswered immediately after the baseline encourage students to remain in the course to learn the answers.

The Baseline Diagnostic Quiz is placed in the first unit of the course. It is ungraded and responses are optional. It asks the same questions as the diagnostic quiz in intervention 1; however in this case, the answers will not be shown to the students.

Control/Placebo:
The control group will receive a placebo survey that is non-technical and asks about demographics, aspirations, and professional background. The Placebo Survey is placed in the first unit of the course. It is ungraded and responses are optional.

Experimental design

Upon entering the very first module, participants (online students) are randomly assigned by the edX software into one of three groups:
Group A receives: Intervention 2: Baseline Diagnostic Quiz without answers
Group B receives: Intervention 1: Baseline Diagnostic Quiz with answers
Group C is the control group

Design details

Randomization method
edX software automatically conducts the randomization.
Randomization unit
The randomization unit is at the participant (student) level. Each participant can be independently randomly assigned to one of the three gruops.
Clusters
4200
Observations
4200
Arms
planned: 1400 receives intervention 1 (baseline with answers), 1400 receives intervention 2 (baseline without answers), 1400 control Because each student is randomly assigned when they enter the first module, these numbers are estimates.
Primary outcomes
Percent of course completed Number of people who open module 2 (post-baseline) Number of people who complete module 2 (post-baseline) Total score

Cited by

No citation data available for this registration yet.

Keywords