Bundled behavioral messages and course completion among online adult learners
How to cite
Diaz-Contreras, Jhon; Munoz-Herrera, Manuel. 2026. "Bundled behavioral messages and course completion among online adult learners." AEA RCT Registry. AEARCTR-0018526. https://doi.org/10.1257/rct.18526-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This randomized controlled trial evaluates whether a bundle of behavioral messages, delivered during the rollout of an online training program, can increase course completion among online adult learners in Colombia. The study targets adult students who register for a fully asynchronous self-efficacy course composed of nine sessions and randomly assigns them to one of two groups: (1) a control group (NOINFO) that receives only minimal session-launch notifications during course delivery; and (2) a treatment group (INFO) that receives, for each new session, an email bundling a personalized commitment reminder referencing the student’s prior decision to enroll, information about the labor-market value of certified self-efficacy skills, and social proof in the form of completion counts and average ratings of the same session in a previous cohort. The primary outcome is course completion (completing all nine sessions before the deadline). Secondary outcomes include the number of sessions completed, session-level completion, a quality-of-engagement measure based on whether the student watched each session’s video in full, and self-reported recall and perceived influence of the different components of the INFO emails on the decision to complete the course.
Intervention
This study evaluates the effect of a bundle of behavioral messages on the completion of an online training program focused on self-efficacy, offered to adult students enrolled in virtual undergraduate programs at a university in Colombia. The course is fully asynchronous and free of charge, comprises nine sessions of approximately 30 minutes each (4.5 hours total), and grants a certificate of completion plus entry into a lottery for cash prizes to students who finish all sessions before the deadline. Invitations are sent and registration is handled directly by the university through its internal outreach channels.
After the registration window closes, registered students are randomly assigned to one of two groups for the course-delivery phase. The intervention varies the content of the session-launch emails sent each time a new session is released:
• NOINFO (Control): Students receive a minimal session-launch notification each time a new session is released. The email informs them that a new session is available and provides the access link, with no additional motivational or informational content. The email also includes a personalized summary of the student’s progress (which sessions they have already completed) and a PDF with a synthesis of conclusions from prior sessions.
• INFO (Treatment): Students receive a session-launch email that bundles three behavioral elements: (i) a personalized commitment reminder referencing the student’s prior decision to enroll (e.g., “Tu compromiso con este proceso sigue vigente”, “Asegúrate de sacarle provecho a la decisión que tomaste al inscribirte”); (ii) information about the labor-market returns to certified self-efficacy skills (e.g., wage premium for certified candidates, value of credentials as a signal of self-management to employers, role of certificates as filters in résumé screening); and (iii) social proof from a previous cohort, reporting how many students completed the same session and the average rating they assigned to it. The email also includes a personalized summary of the student’s progress (which sessions they have already completed) and a PDF with a synthesis of conclusions from prior sessions.
The differential treatment is delivered for sessions 1 through 9 of the course (eight emails per student). Session 0 launches simultaneously with registration and is identical across arms.
Experimental design
This study is implemented as a randomized controlled experiment (RCT). The target population consists of all students who register for an online self-efficacy course offered to students enrolled in virtual undergraduate programs at a university in Colombia. Based on administrative records, this population is predominantly adult (median age 30), with a diverse age distribution, and is characterized by balancing work, family, and study obligations.
Once the one-week registration window closes, registered students are randomly assigned at the individual level to one of two groups:
• NOINFO (Control): Receives a minimal session-launch notification each time a new session is released, with no additional motivational or informational content.
• INFO (Treatment): Receives, for each new session, a session-launch email that bundles a personalized commitment reminder, information about the labor-market value of certified self-efficacy skills, and social proof reporting completion counts and average ratings of the same session in a previous cohort.
The differential treatment is delivered for sessions 1 through 9. Session 0 is identical across arms. For the analysis, we will estimate OLS regressions of the primary and secondary outcomes on a binary indicator for INFO, with and without controls for stratification variables and baseline covariates from the registration survey. We will present results from one-sided tests to improve statistical power, given the directional nature of our hypothesis.
Primary hypothesis:
The bundle of behavioral messages increases course completion relative to the control group (INFO > NOINFO). We expect the bundle to operate by sustaining the salience of the original enrollment decision, raising the perceived labor-market value of finishing the course, and reducing belonging uncertainty by signaling that comparable students complete the same sessions.
Secondary and exploratory analyses:
We will conduct exploratory analyses to examine heterogeneous treatment effects using available baseline characteristics (e.g., gender, GPA, among others). We will also examine the mechanisms underlying any detected effects through the secondary outcomes measured in the registration survey.
In addition, we will use the post-course survey items measuring recall and perceived influence of each component of the INFO emails to explore which element of the bundle (the personalized commitment reminder, the information on the labor-market value of the certificate, or the social proof from a previous cohort) was most associated with the decision to complete the course. We do not have an a priori prediction about which of the three components will be most effective; the relative importance of each component will therefore be assessed in a purely exploratory manner.
Design details
Cited by
No citation data available for this registration yet.