Learning with Machines: A Randomized Evaluation of AI-Assisted Education in Schools
How to cite
Liu, Elaine; Zuo, Sharon Xuejing; Chen, Zhao; Li, Kaiyue; Zhong, Yuelin. 2026. "Learning with Machines: A Randomized Evaluation of AI-Assisted Education in Schools." AEA RCT Registry. AEARCTR-0018533. https://doi.org/10.1257/rct.18533-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study evaluates the effectiveness of a technology-based learning intervention in rural middle schools using a randomized controlled trial. A group of students receives access to a structured learning tool designed to support independent study, while a comparison group continues with their standard study routine. Both groups are assessed regularly throughout the study period.
Our primary outcome of interest is performance on a standardized examination taken at the end of the study period. We also collect data on student engagement and learning behavior, as well as on classroom instruction. Where possible, we benchmark outcomes against students in urban areas.
Intervention
The intervention provides structured AI-assisted tutoring sessions to ninth-grade students in eight rural middle schools. Students assigned to the treatment group attend a daily two-hour session using an AI chatbot (Doubao) on tablets, following a fixed six-step structure: (1) a short pre-test on the day's topic, (2) AI-generated diagnostic feedback based on the pre-test, (3) an extended period of personalized AI-guided practice in which problem difficulty adapts to student performance, (4) an AI-generated summary of the key concepts covered, and (5) a post-test on the same topic. Students receive a small token incentive, redeemable for a gift, for completing all required steps of each session. Research assistants are present to monitor session integrity and record participation.
Students in the control group continue with their regular classroom instruction and complete the same pre- and post-tests as the treatment group, without access to the AI tutoring sessions, allowing learning gains to be compared across groups. The intervention is delivered over several weeks in the lead-up to the high school entrance examination (zhongkao), with schools entering the study on a staggered schedule."
Experimental design
This is an individually randomized controlled trial conducted in eight rural middle schools among ninth-grade students preparing for the zhongkao (high school entrance examination). The intervention takes place during the evening self-study period, a block of time with no formal classroom instruction, during which students are otherwise expected to study independently. Within each school, students are randomly assigned to treatment or control, stratified by baseline test score tertile and gender. Students assigned to treatment use this self-study time for a structured two-hour AI-assisted tutoring session with an AI chatbot on tablets, delivered by research staff. Students assigned to control spend this same evening self-study period as they normally would: studying independently, with a teacher available in the room to answer questions if a student asks, but without any structured instruction or AI access. Both groups complete the same periodic assessments. Schools enter the study on a staggered schedule between mid-April and mid-May 2026, so students accumulate varying amounts of exposure time before the exam. The main comparison of interest is between treatment and control students' performance on the zhongkao. A separate, non-randomized comparison sample of students in two urban schools was also surveyed to help interpret rural-urban differences in outcomes and AI access, but is not part of the randomized comparison.
Design details
Cited by
No citation data available for this registration yet.