The Tailoring Premium: How AI Design Unlocks Student Engagement and Learning

How to cite

Polanco-Jimenez, Jaime; Witte, Kristof De. 2025. "The Tailoring Premium: How AI Design Unlocks Student Engagement and Learning." AEA RCT Registry. AEARCTR-0015266. https://doi.org/10.1257/rct.15266-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jaime Polanco-Jimenez* , Kristof De Witte

* Primary investigator

Registry ID
AEARCTR-0015266
DOI
https://doi.org/10.1257/rct.15266-1.1
First registered
Last updated
Status
Completed
Trial dates
2025-01-30 – 2025-04-30
Countries
Belgium (Flemish Region)
JEL codes
I20 I21 I22 I24 C93 O15
Registry record
http://www.socialscienceregistry.org/trials/15266

Abstract

We provide the first causal estimate of the ``tailoring premium'' for educational AI. In a randomized trial with 2,440 secondary students, we demonstrate that offering a curriculum-tailored chatbot increases immediate learning by 0.126 standard deviations, while a generic chatbot produces learning gains that are statistically indistinguishable from traditional instruction. The learning gain is driven by a 13.2 percentage point increase in module completion, demonstrating that the tool's value comes from solving student engagement. For compliers—students induced to complete the module by the tailored design—the effect is larger and more durable, increasing long-term knowledge retention by a significant 0.309 standard deviations. The returns to educational AI depend critically on its design, not just its availability.

Intervention

Traditional Instruction: Students receive standard financial literacy instruction using traditional methods and materials.
Group 1 (Control Group). These students receive a classic learning path with instructions and use a spreadsheet or calculator.
Group 2 (Treatment Group 1). This group receives a reduced learning path where they have to find the answers in a general-purpose AI.
Group 3 (Treatment Group 2). This group follows instructions from a tailored-AI chatbot that adjusts the questions and instructions to the answers of the students.

Experimental design

This study uses a randomized controlled trial (RCT) design to evaluate the impact of different learning approaches on financial literacy among secondary school students. Participants will be randomly assigned to one of three groups: a control group receiving standard instruction, a treatment group using general-purpose AI for learning support, and a second treatment group receiving a different form of AI-enhanced instruction. All students will receive instruction on financial literacy topics. Key measures, including learning performance and student experiences, will be assessed at various points during the study.

Design details

Randomization method
Students will be assigned to one of the three learning groups using a random number generator within the LimeSurvey platform. Once a student completes the baseline survey in LimeSurvey, the platform will automatically assign them to a group using a pre-programmed random number sequence, ensuring each student has an equal chance of being assigned to any of the three groups.
Randomization unit
Individual - Since students are being assigned individually
Clusters
0
Observations
Based on the power analysis we expect the participation of more than 732 students.
Arms
244
Primary outcomes
Learning Performance Attitude and Motivation Learning Experience & User Experience Self-Confidence & Self-Efficacy
Secondary outcomes
Heterogeneities by gender, vocational school, language spoken at home, previous knowledge of AI...

Cited by

No citation data available for this registration yet.

Keywords