AI in the Classroom: Evaluating the Impact of Teacher Training on Practices and Student Outcomes

How to cite

Lopez, Carolina; Zarate, Roman Andres; Molina, Ezequiel. 2025. "AI in the Classroom: Evaluating the Impact of Teacher Training on Practices and Student Outcomes." AEA RCT Registry. AEARCTR-0016336. https://doi.org/10.1257/rct.16336-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Carolina Lopez* , Roman Andres Zarate , Ezequiel Molina

* Primary investigator

Registry ID
AEARCTR-0016336
DOI
https://doi.org/10.1257/rct.16336-1.0
First registered
Last updated
Status
On going
Trial dates
2025-01-01 – 2026-12-01
Countries
Private
JEL codes
C93, I21
Registry record
http://www.socialscienceregistry.org/trials/16336

Abstract

Can artificial intelligence (AI) improve teaching practices and student learning in developing countries? This study evaluates whether training primary school teachers in Peru to integrate Large Language Models (LLMs) into their daily work can enhance instructional quality and student outcomes. We conduct a large-scale, multi-arm, field experiment across public schools in Lima, Peru. The main intervention is a structured teacher training program focused on the use of AI for educational practices. In addition to the primary treatment, the study includes two cross-randomized variations. The first compares two modes of ongoing support—peer-based networks versus authority-led guidance—to assess which approach better sustains AI adoption. The second introduces a complementary training that emphasizes tailoring instruction to students’ learning levels, following “Teaching at the Right Level” (TaRL) principles. By examining both implementation and impacts on teaching practices and student learning, this study provides timely evidence on the potential for AI to equitably and effectively support educators and learners in low- and middle-income contexts.

Intervention

The intervention evaluates a teacher training program designed to integrate Large Language Models (LLMs) into educational practices. The program has been developed in close collaboration with local teachers and is tailored to the Peruvian context to ensure relevance and practical application.

The training begins with an initial eight-hour in-person session covering:
1. AI literacy fundamentals
2. Principles of responsible AI use
3. Hands-on practice with real-world classroom scenarios. These scenarios, developed through a teacher innovation contest in Lima, include:
3.1. Adapting lesson plans for diverse student levels
3.2. Creating engaging learning materials in Spanish
3.3. Designing formative assessments aligned with Peru’s curriculum
3.4. Drafting effective parent communications
3.5. Streamlining administrative tasks

In addition to the main intervention, the program will consider the following subcomponents:
1. Cross-randomized support structures: To sustain and reinforce the program’s impact, the intervention explores the effectiveness of two types of ongoing support:
1.1. Peer-based support: Teachers are organized into WhatsApp groups moderated by teachers who encourage weekly participation.
1.2. Authority-led support: Similar WhatsApp groups are monitored by education supervisors designated by the regional education authority (DRELM).

This nine-month continuous reinforcement phase includes regular check-ins, troubleshooting sessions, and active documentation of teacher experiences. Real-time barriers to adoption are identified and addressed to ensure the program’s effectiveness and scalability.

2. Teaching at the Right Level (TaRL) subtreatment: A second cross-randomized component focuses on differentiated instruction. A subset of teachers will receive additional online training in Teaching at the Right Level (TaRL) principles, emphasizing how to use AI tools to tailor instruction to students’ starting learning levels and promote individualized support. The comparison group will receive general AI training without this targeted emphasis on instructional differentiation.

Experimental design

The randomization was conducted in three phases, all stratified by district to ensure balance across treatment arms.

Phase 1: Network-Level Randomization
A network consists of approximately six geographically proximate schools that share a supervisor and are grouped for administrative purposes by DRELM. To avoid spillovers within networks, 143 networks were assigned to the pure control group, while 84 networks were selected for the school-level randomization.

Phase 2: School-Level Randomization
Within the selected networks, 390 schools were randomized at the school level, stratified by district and by prior performance (high- or low-performing). A total of 196 schools were assigned to the main treatment group, AI in person training for teachers, and 194 to the control group.

Phase 3: Cross-Randomized Sub-Treatments
Two sub-treatment arms were randomized at the school level:
Support Structure: Schools were assigned to either a peer-based network or authority-led guidance. In the peer-based arm, a teacher will monitor and encourage weekly participation among their peers. In the authority-led arm, this role is taken by a supervisor designated by the DRELM. A total of 97 and 99 schools were assigned to the peer- and authority-led treatment arms, respectively.

Teaching at the Right Level (TaRL): One group of teachers receives additional online training focused on the role of AI tools in tailoring instruction to students’ initial learning levels, promoting differentiated instruction based on students’ diverse abilities. The comparison group receives general AI training for classroom practices, with less emphasis on adapting content to individual learning levels. A total of 97 and 99 schools were assigned to the general vs. TaRL treatment arms, respectively.

This multi-level, stratified randomization strategy is designed to rigorously evaluate both the overall effect of AI training and the relative effectiveness of alternative support and instructional approaches.

Design details

Randomization method
The randomization was conducted using a computer. Each phase of the randomization process was executed 100 times, and we selected the randomization iteration that achieved the best balance across treatment arms using a max-min p-value criterion. This method maximizes the smallest p-value across covariates to ensure the best possible balance on observable characteristics. Randomization was stratified by district and conducted in three phases: 1. Network-Level Randomization: Entire school networks were randomly assigned to either the pure control group or the eligible treatment pool to minimize spillovers within networks. 2. School-Level Randomization: Within selected networks, schools were randomly assigned to the main treatment (AI training) or the control group. 3. Cross-Randomized Sub-Treatments: Schools in the treatment group were further randomized into: 3.1. Peer-based versus authority-led support structures. 3.2. General AI training versus Teaching at the Right Level (TaRL)–focused training. This multi-level, stratified randomization strategy ensures rigorous internal validity and balance across treatment arms.
Randomization unit
The randomization unit was the school.
Clusters
1. 143 networks of 4.7 schools on average: 84 networks are part of the sample and 59 networks are pure control.-- 2. Within the 84 networks of the same, there is a total of 390 schools.
Observations
We plan to have around three teachers per school for a total of 1,170 teachers, and around 90 students per school for a total of 35,100 students.
Arms
The 390 schools were classified as follows: 1. 194 control schools 2. 196 treated schools: 2.1. 47 schools assigned to peers and general. 2.2. 49 schools assigned to peers and tatrl. 2.3. 50 schools assigned to authority and general. 2.4. 50 schools assigned to authority and tatrl.
Primary outcomes
1. Teachers’ time use and their practices both inside and outside the classroom. 2. Students' standardized scores in math and reading comprehension for 6th grader students.
Secondary outcomes
Secondary Student Outcomes: 1. Student academic progress based on administrative records. Teacher secondary outcomes: 1. AI engagement metrics, including frequency of use and types of AI tools adopted, potentially complemented by anonymized usage data from Microsoft accounts (subject to final data-sharing agreements).
Partners
Name: Dirección Regional de Educación Lima Metropolitana Partner_type: government Url: https://www.gob.pe/regionlima-drelm Name: World Bank Partner_type: none Url: https://www.worldbank.org/ext/en/home

Cited by

No citation data available for this registration yet.

Keywords