Adopting Computer Assisted Learning (CAL) at Scale: Training and Supporting Teachers and Families to Use CAL Technologies in Puerto Rico Public Schools
How to cite
Bobonis, Gustavo J.; Oreopoulos, Philip; Sotomayor, Orlando; Zarate, Roman Andres. 2021. "Adopting Computer Assisted Learning (CAL) at Scale: Training and Supporting Teachers and Families to Use CAL Technologies in Puerto Rico Public Schools." AEA RCT Registry. AEARCTR-0007720. https://doi.org/10.1257/rct.7720-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Educators and policymakers are facing growing disparities in academic achievement between high and low-income students, differences that appear to have increased as a result of the Covid-19 pandemic. A promising cost-effective strategy to address this issue is the use of Computer Assisted Learning (CAL) technologies, which help students progress through topics at their own pace while receiving feedback and support. However, whether CAL interventions can be scaled up effectively remains a first-order question. Our project is a collaboration with the Puerto Rico Department of Education to experimentally evaluate the impact of a large-scale training and coaching program to support mathematics teachers to use Khan Academy, one of the most popular mathematics-oriented CAL educational platforms. The program will be gradually phased-in and offered at scale to the population of fourth to eighth grade Mathematics teachers in the system’s approximately 650 primary and middle schools. We will also test whether complementary technology-enabled behavioral interventions can promote its use via parental involvement. The study will examine the program’s impact on math achievement in elementary and secondary school classrooms, as well as student, teacher, and parent experiences with the program.
Intervention
The Puerto Rico Department of Education (PRDE) is seeking to introduce teaching reforms in Mathematics to improve educational outcomes in the territory. The study will rigorously evaluate - via a randomized controlled trial - the impact of the ATEMA (Aplicación de la tecnología en la enseñanza de matemáticas) Computer Assisted Learning (CAL) program. The ATEMA program is a teacher training and coaching program designed to support math teachers to use one of the most popular mathematics-oriented CAL educational platforms, Khan Academy (KA). The program will be gradually phased-in and offered at scale to the overall population of approximately 1,630 fourth to eighth-grade math teachers in the PRDE’s approximately 670 primary and middle schools over a three-year period (Academic Years 2021-22 through 2023-24). The overarching objective is to implement a scalable program, designed to function well whether classes are taught online or in person, to improve student achievement and experience in a manner which is free, easy to use, encourages engagement, and which also improves teacher satisfaction and workload.
We will support the PRDE’s development of and test an in-class or virtual program that incorporates KA into in-person or online grades 4-8 math classrooms, particularly as a tool for reinforcing classroom instruction and greater personalization. The training and support of each cohort of teachers will take place over the course of one academic year (approximately 20 hours), and will comprise of three types of sessions: (i) an initial six-hour session before the start of the academic year; (ii) a second session focused on incorporating KA into the class content; and (iii) 18 weekly sessions of 20 to 30 minutes. The program will also include a second treatment component that will establish teacher-parent communication with actionable suggestions, based on KA administrative data, to increase students’ use of and engagement with the platform.
Experimental design
The experimental design comprises two stages. Approximately 670 PRDE schools were stratified by academic level (i.e., primary, middle school), PRDE region, and performance (below or above the median school) at baseline. All the schools in Puerto Rico with students in 4th- to 8th-grade are part of the analysis. All schools within each stratum were randomly assigned to one of five main experimental arms:
● Treatment Arm 1 (TA1): Cohort 1 — ATEMA program (AY 2021-22)
● Treatment Arm 2 (TA2): Cohort 1 — ATEMA program + Parental Engagement Emails (AY 2021-22)
● Treatment Arm 3 (TA3): Cohort 2 — ATEMA program (AY 2022-23)
● Treatment Arm 4 (TA4): Cohort 2 — ATEMA program + Parental Engagement Emails (AY 2022-23)
● Control Group (Control): Cohort 3 – Status Quo
Each of the school cohorts in this stepped-wedge design is composed of 223-224 schools, with approximately 112 schools in each one of TA1-TA4. Eligible teachers in each school will be invited to enroll in ATEMA, and we will engage school principals, regional math administrators, and regional superintendents to encourage their staff to participate. The randomization of the supplementary parental information and engagement intervention was also performed at the school level: parents of students in half of the schools eligible for the ATEMA program (112 schools) in cohorts 1 and 2 have received or will receive the parental engagement treatment each year.
Design details
Cited by
No citation data available for this registration yet.