Online Learning and Labor Market Outcomes: Experimental Evidence from Colombia

How to cite

Zarate, Roman Andres; Majerowicz, Stephanie. 2022. "Online Learning and Labor Market Outcomes: Experimental Evidence from Colombia." AEA RCT Registry. AEARCTR-0010016. https://doi.org/10.1257/rct.10016-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Roman Andres Zarate* , Stephanie Majerowicz

* Primary investigator

Registry ID
AEARCTR-0010016
DOI
https://doi.org/10.1257/rct.10016-1.1
First registered
Last updated
Status
Completed
Trial dates
2020-09-01 – 2022-06-30
Countries
Colombia (Country-wide)
Registry record
http://www.socialscienceregistry.org/trials/10016

Abstract

This project studies whether workers affected by the pandemic benefited from access to free certificates and degrees provided by massive open online courses (MOOCs). One of the largest and most recognized MOOC providers offered governments in Latin America free certificates during the first outbreak of Covid-19 in the region. As part of this program, a government agency in Colombia conducted an RCT where 10,000 beneficiaries of student loans received the opportunity to enroll and certify their skills in MOOCs. The project will study participants' behavior on the platform, including selection, enrollment, and completion of online courses. Furthermore, it will track participants to the labor market to study the impact of MOOCs on formal employment and wages during and after the pandemic.

Intervention

The central intervention is providing free MOOC certificates. To ease the negative consequences of the pandemic on the labor force, a major MOOC provider launched an initiative, in which Latin American governments could apply for free MOOC certificates for their citizens. A government agency in Colombia applied for and received 10,000 slots.

Participants who were selected for treatment could enroll in as many courses as they wanted from a catalog with more than 3,800 courses. As long as they complete all the requirements, they could obtain the certificate of completion for free before a deadline stipulated by the MOOC provider. Eligible participants could also enroll in “specializations,” which are higher-level certificates, usually composed of four different courses. The participants had approximately three months to complete the courses.

The MOOC provider also encouraged all the institutions to run a campaign to promote course enrollment and completion. The government agency in charge fulfilled this requirement and constantly sent emails to participants encouraging enrollment and completion. As part of this program, they also promoted the most-demanded courses in the region to all eligible participants, for instance, by sending emails to participants where they highlighted specific courses.

Experimental design

The research design was a simple randomization at the individual level, without any stratification. Among the approximately 23,000 applicants to the program, 10,000 were randomly selected to be eligible to participate in the program, and the rest were placed in the control group.

The government agency in charge of the certificates conducted the randomization with the technical support of the research team. The randomization took place in October of 2020. About 7,000 of the applicants who received an initial offer, signed-up for the program. As it was of the interest of the government agency to maximize take-up, they performed a second-round randomization where 3,000 applicants in the control group received a second-round offer to join the program.

Participants who were selected for treatment could enroll in as many courses as they wanted out of 3,800 eligible courses, and could receive as many certificates certifying their skills as they wanted as long as they completed the courses by the deadline established by the MOOC provider.

Design details

Randomization method
The research design was a simple randomization at the individual level, without any stratification. The government staff conducted the randomization on a computer with technical support from the research team. Among the approximately 23,000 applicants to the program, 10,000 were randomly selected to be eligible to participate in the program. The research team suggested to the government agency to re-randomize to avoid chance imbalances, following Banerjee et al. (2017). The randomization was run 100 times, and balance checks were performed on variables including demographic characteristics, eligibility for safety net programs, debt in student loans, and education variables for a total of 36 variables. The code used a max-min p-value criteria, keeping the randomization with the largest minimum p-value among the 36 variables used for balance checks.
Randomization unit
The randomization was done at the individual level.
Clusters
There are 23,000 individuals, the treatment was randomized at the individual level and was not clustered at any other level.
Observations
The observations will be the 23,000 individuals, 10,000 of whom were randomized into the treatment group and the rest into the control group.
Arms
Among the approximately 23,000 applicants to the program, 10,000 were randomly selected to be eligible to participate in the program, while the rest make up the control group.
Primary outcomes
The goal of the experiment is to assess the impact of access to formal certificates on the platform on labor market outcomes. We will use social security administrative data that all employers in the formal sector report every month for health and pension contributions. The data allows us to identify whether an individual is formally employed, the sector in which they are employed, and the wage.
Secondary outcomes
While the primary analysis will focus on the treatment effects of free online certificates on labor market outcomes, we will also study how individual variables affect course selection and completion. We will also use machine learning and a diff in diff strategy to estimate heterogeneous treatment effects by course registration and completion.

Cited by

No citation data available for this registration yet.

Keywords