Experimental evidence on data-driven remedial instruction in the Dominican Republic

How to cite

Lopez, Carolina; Pineda, Astrid. 2023. "Experimental evidence on data-driven remedial instruction in the Dominican Republic." AEA RCT Registry. AEARCTR-0012383. https://doi.org/10.1257/rct.12383-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Carolina Lopez* , Astrid Pineda

* Primary investigator

Registry ID
AEARCTR-0012383
DOI
https://doi.org/10.1257/rct.12383-1.0
First registered
Last updated
Status
On going
Trial dates
2023-09-15 – 2026-06-30
Countries
Dominican Republic ()
JEL codes
C93, I21, I24, O15
Registry record
http://www.socialscienceregistry.org/trials/12383

Abstract

Learning losses due to COVID-19 have been substantial, especially for students coming from low socioeconomic backgrounds, further worsening existing learning deficits in many developing countries. To address these losses, the use of tutoring and computer-assisted instruction holds promise for accelerating learning recovery. However, there is limited knowledge on how education systems can effectively implement these approaches at scale. Computer adaptive learning (CAL) softwares are particularly noteworthy for their ability to cater to students' individual learning levels. However, most evidence on CAL is based on after-school settings and primary-school-aged children, making it challenging to extrapolate to older students or in-school settings. Tutoring, while effective, faces scalability challenges due to cost and availability of qualified tutors. In this proof-of-concept, we aim to evaluate an in-school intervention combining CAL with group tutoring as a potentially more scalable alternative to accelerate learning recovery among teenagers. Moreover, we aim to generate evidence on whether using CAL-generated data for targeted tutoring and teacher support can lead to better student outcomes. We plan to use in-depth data, including on teaching practices and teachers’ time use, to investigate the underlying mechanisms behind the observed changes.

Intervention

In this proof-of-concept, we aim to evaluate an in-school intervention combining the use of a Computer Adaptive Learning (CAL) software with group tutoring as a potentially scalable alternative to accelerate learning recovery among teenagers. Moreover, we aim to generate evidence on whether using CAL-generated data for targeted tutoring and teacher support can lead to better student outcomes. The intervention has two components:

Component #1 introduces the use of CAL as part of schools’ regular instruction, substituting two out of seven weekly hours scheduled for math instruction. The software selected has been adapted to the Dominican Republic’s math curriculum. This arm includes “CAL aides” to support teachers in leveraging the rich source of data the platform provides to assess student learning and adapt their instruction. CAL aides will meet for one hour every two weeks with teachers to support them in accessing and monitoring students’ assessments, tracking their progress, identifying topics that students do not yet master for lesson planning, and using resources offered by the software to ease their workload (e.g., assigning and grading homework and tests).

Component #2 introduces targeted, data-driven (based on CAL software data) small-group tutoring. This component aims to explore complementarities between CAL and more
traditional forms of instruction, such as tutoring, by providing up to three weekly hours of tutoring to high-need students. Students participating in tutoring sessions will have access to the platform (component #1), and tutors will be instructed to use the data to monitor student progress, identify learning gaps, and plan their lessons. These lessons will be implemented during school hours devoted to extracurricular activities.

Experimental design

Classroom-level assignment, participating classrooms will be randomly assigned to two groups:
1. Treatment. Students and teachers in selected classrooms will receive access to the CAL software and general training on its use (component #1).
2. Control group. The control group will not receive access to the CAL software..

Our design would follow a matched pair randomization design, where classrooms would be matched based on their size. One of each classroom in a pair would be randomly assigned to treatment. In addition, among classes in the treatment arm, we will cross-randomize component #2 (covering 20% of the students in each selected classroom with the lowest math performance).

Each classroom will receive one of three treatment combinations: no intervention (pure control), CAL software without tutoring, and CAL software with tutoring for students with the lowest math performance (bottom 20%).

Design details

Randomization method
Randomization was conducted in the office using Stata.
Randomization unit
The unit of randomization is the classroom.
Clusters
~40 classrooms
Observations
~1300 students attending 9th grade.
Arms
~20 classrooms in the control group and ~20 classrooms in the treatment group (CAL treatment). Among the schools in the treatment group, ~10 classrooms will be selected to receive tutoring, which covers 20% of the students with the lowest math performance (within each classroom).
Primary outcomes
Our pilot includes a data collection strategy to measure final and intermediate students’ and teachers’ outcomes to better understand the mechanisms underlying the results. For this, we will administer surveys among students and teachers at baseline and endline, as well as student assessments. Testing these instruments at this stage will help us to define the best approach to measure our main outcomes in the scale-up phase. • Student outcomes: We will evaluate students’ learning outcomes in math through standardized assessments (endline survey) and will track progress using the CAL platform’s periodic assessments (time using the platform, number of mastered topics). • Teacher outcomes: We want to assess whether the software can effectively blend with regular instruction and improve teachers’ productivity. We will focus on measuring teachers’ time use and their practices in and outside the classroom (endline survey, for example “How many hours did you spend at planning lessons/correcting tests and assignments…?, “Indicate how often the following occurred: You talked to a parent about their child's performance/You led a training workshop for your colleagues at your school,” etc.).

Cited by

No citation data available for this registration yet.

Keywords