Scaling up virtual learning of online learning in high schools

How to cite

McKenzie, David; Asanov, Igor. 2020. "Scaling up virtual learning of online learning in high schools." AEA RCT Registry. AEARCTR-0005982. https://doi.org/10.1257/rct.5982-1.11

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

David McKenzie* , Igor Asanov

* Primary investigator

Registry ID
AEARCTR-0005982
DOI
https://doi.org/10.1257/rct.5982-1.11
First registered
Last updated
Status
Completed
Trial dates
2020-05-18 – 2023-05-31
Countries
Ecuador ()
JEL codes
O12, O14, M31, L26
Registry record
http://www.socialscienceregistry.org/trials/5982

Abstract

We have an ongoing evaluation that tests the impacts of a training program that uses online learning for 15-17 years old high school students, with a focus on teaching entrepreneurial skills and a scientific mindset. That program was taught using computers in schools, during school time, until schools were closed with the COVID-19 pandemic, after which students continued to finish the material from their homes. With the COVID-19 pandemic closing schools, school systems have been looking for content and ways to engage students. We therefore are testing the impact of launching this material to another region of the country. We are interested in two key questions:
1) How can we best encourage take-up and usage of online learning materials among high school students during the COVID-19 school closures? This will be tested by comparing the impact of different nudges, reminders, and other interventions to encourage take-up and usage..
2) What is the impact of this program on learning outcomes? This will involve comparing impacts for students who are assigned to the personal initiative and negotiations classes to those who receive other courses. We will then also compare the learning gains from this virtual learning to those from students in our first experiment, who did most of the learning under teacher supervision in schools.

Intervention

The intervention is provided in two phases according to the educational regimes in Ecuador: Highlands Educational Regime (Students Finish the program in Summer 2020) and Coastal Educational Regime (Students Finish the program in Winter 2020). We have two types of interventions: the training interventions, and then interventions to enhance the take-up of training.

Highlands Educational Regime (Students Finish the program in Summer 2020) includes:
Training interventions.
Schools are randomly assigned to one of the following four treatment groups:
1) Personal initiative + Negotiations training + Role Model videos
2) Personal initiative + Negotiations training + Placebo videos
3) Spanish + Statistics + Role Model videos
4) Spanish + Statistics + Placebo videos

Interventions to enhance the take-up of training.
We are testing different interventions to encourage take-up and usage of the online platform on different levels: Students and teachers. These include on student level:
i) control group
ii) lottery reminder to students - students earn a ticket for a lottery prize for each completed module
iii) Encouragement to students to find a peer in the house to plan and study together
iv) Encouragement to students for those who complete their work
These include on teachers level:
I)control
ii) Role model testimonials/motivational videos to teachers from teachers and students who have already completed the course
iii) Reminder about monitoring and deadline to teachers

Coastal Educational Regime (Students Finish the program in Winter 2020) that includes:
Training interventions.
Schools are randomly assigned to one of the following 6 treatment groups:
1) Personal initiative + Negotiations training + Role Model videos
2) Personal initiative + Negotiations training + Placebo videos
3) Spanish + Statistics + Role Model videos
4) Spanish + Statistics + Placebo videos
3) Spanish + English + Role Model videos
4) Spanish + English + Placebo videos

Interventions to enhance the take-up of training.
We are testing different interventions to encourage take-up and usage of the online platform on the school level:
i) Control
ii)Monitoring (information)
iii)Monitoring (information) + lottery reminder to students - students earn a ticket for a lottery prize for each completed module and performance on platform.
iv)Monitoring (information) + Encouragement to team-up with peers remotely.

Experimental design

Course type is randomized at the school level to one of the following in the Highlands Educational Regime:
1) Personal initiative + Negotiations training + Role Model videos
2) Personal initiative + Negotiations training + Placebo videos
3) Spanish + Statistics + Role Model videos
4) Spanish + Statistics + Placebo videos

Take-up interventions are randomized at the individual level or class/teacher level, depending on the intervention.

Course type is randomized at the school level to one of the following in the Coastal Educational Regime:
1) Personal initiative + Negotiations training + Role Model videos
2) Personal initiative + Negotiations training + Placebo videos
3) Spanish + Statistics + Role Model videos
4) Spanish + Statistics + Placebo videos
3) Spanish + English + Role Model videos
4) Spanish + English + Placebo videos

Take-up interventions are cross-randomized on the school level.

Design details

Randomization method
Randomization done in office by computer or on the online platform
Randomization unit
School for main interventions Class for some take-up interventions Student for individual interventions
Clusters
Highlands Educational Regime (Students Finish the program in Summer 2020) includes: 416 schools 532 classes Approximately 24,000 potential students Coastal Educational Regime (Students Finish the program in Winter 2020) that includes: 598 schools 795 classes Approximately 28,000 potential students
Observations
We will only observe outcomes for students who log in once to the platform - which we anticipate to be over 10,000 students in Highlands Educational Regime and over 11, 000 in Coastal Educational Regime
Arms
Highlands Educational Regime (Students Finish the program in Summer 2020) includes: 104 schools in each treatment group 127 to 136 classes in each treatment group Coastal Educational Regime (Students Finish the program in Winter 2020) that includes: 1) 149 schools Personal initiative + Negotiations training + Role Model videos (~200 classes) 2) 149 schools Personal initiative + Negotiations training + Placebo videos (~200 classes) 3) 75 schools: Spanish + Statistics + Role Model videos (~100 classes) 4) 75 schools: Spanish + Statistics + Placebo videos (~100 classes) 3) 75 schools: Spanish + English + Role Model videos (~100 classes) 4) 75 schools: Spanish + English + Placebo videos (~100 classes)
Primary outcomes
We have two sets of outcomes: take-up outcomes, and learning outcomes. Take-up outcomes: 1) Proportion of students who complete at least one module 2) Number of modules completed by students 3) Total days spent active on platform Learning outcomes: Test scores of learning based on Spanish and English literacy, knowledge of statistics, science, negotiations and personal initiative.

Papers from this trial

  1. Asanov I, Asanov AM, Åstebro T, Buenstorf G, Crépon B, McKenzie D, Flores T FP, Mensmann M, Schulte M. (2023) System-, teacher-, and student-level interventions for improving participation in online learning at scale in high schools. Proceedings of the National Academy of Sciences (PNAS). 120(30):e2216686120 https://doi.org/10.1073/pnas.2216686120

Cited by

No citation data available for this registration yet.

Keywords