The Personalization Trap: Does AI Tutoring Build Learners or Dependence?
How to cite
Lopez, Carolina; Molina, Ezequiel; Reyes, Germán. 2026. "The Personalization Trap: Does AI Tutoring Build Learners or Dependence?." AEA RCT Registry. AEARCTR-0017917. https://doi.org/10.1257/rct.17917-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study evaluates the impact of an AI-powered math tutoring platform and an AI career coaching chatbot on learning outcomes and educational aspirations among 5th-year secondary students in public schools in Lima, Peru. The AI math tutor covers Peru's national math curriculum and is used during regular math class time in school computer labs, supervised by students' regular math teacher. The AI career coach helps students explore post-secondary options, understand economic returns to education, and develop concrete career plans during existing tutoría (guidance) periods.
The study uses a cluster-randomized design with approximately 100 schools as the unit of randomization and three study arms: (1) AI tutoring with learning-science modifications plus career coaching, (2) AI tutoring in standard configuration plus career coaching, and (3) business-as-usual control. The learning-science modifications embed three moments of student cognitive effort (a retrieval practice opener at the start of each session, a predict-reveal-diagnose sequence on errors, and an error detection task on mastered topics) that return to the student metacognitive operations the algorithm otherwise performs on their behalf.
AI tutoring platforms could produce learning gains that are substantially non-durable because the algorithm silently performs metacognitive operations (retrieval, error diagnosis, mastery evaluation) that students need to learn to perform independently—a phenomenon we term the personalization trap. The three modifications test whether returning these cognitive operations to the student makes learning gains persist after platform removal. Primary outcomes are multiple math assessments during the academic year. The study tests whether AI tutoring produces learning gains within existing public-school constraints and whether learning-science modifications to the platform's default interaction design improve the durability of those gains.
Intervention
The study has three arms:
- Arm 1 (AI Tutoring + Career Coach): Students use the same AI math tutoring platform in its standard configuration, without the learning-science modifications. Same content library, same adaptive difficulty algorithm, same session duration, same visual interface. Full AI support throughout. Students also use the AI career coach during tutoría periods, identical to Arm 2.
- Arm 2 (AI Tutoring with Learning-Science Modifications + Career Coach): Students use the AI math tutoring platform with three embedded modifications grounded in the science of learning. Weeks 1-4 are identical to Arm 1 (full AI support); modifications activate in Week 5. (a) Retrieval Opener (~2 min): at the start of each session, 3 problems from the previous session’s topics with no hints, no error diagnosis, no worked examples, just “Correct” or “Incorrect.” (b) Predict-Reveal-Diagnose on Errors (~2 min/session): before the AI shows a worked example, the student predicts their error type (3 buttons); after seeing the worked example, the student articulates their specific error in an open text box. (c) Error Detection on Mastered Topics (~10 sec, once/session): after 3 consecutive correct on a topic, the student is shown a worked solution containing one conceptual error and must identify the error step and describe the error. Students also use the AI career coach during tutoría periods.
- Arm 3 (Business-as-Usual Control): Students continue with regular classroom math instruction and regular tutoría activities, with no access to the AI platform or career coach.
Both treatment arms 1 and 2 use the platform during a portion of regular weekly math hours in the school computer lab, supervised by the regular math teacher.
Experimental design
Cluster-randomized controlled trial with schools as the unit of randomization, conducted in public secondary schools in Lima, Peru. Approximately 110 schools are randomized into three arms: (1) AI tutoring in standard configuration plus career coaching, (2) AI tutoring with learning-science modifications plus career coaching, and (3) business-as-usual control.
The study enrolls approximately 6,600 5th-year secondary students (ages 16-17). Randomization occurs at the school level to avoid contamination between conditions. Schools are selected based on having a functional computer lab, adequate internet connectivity, and sufficient devices, verified by the implementing partner (DRELM, Dirección Regional de Educación de Lima Metropolitana). Data collection includes pre- and post-intervention math assessments, platform usage analytics, and student and teacher surveys, all conducted during regular school hours. The study runs for one full academic year.
Design details
Cited by
No citation data available for this registration yet.