The effect of AI-enhanced instruction on students’ learning, evidence from a randomized controlled trial
How to cite
Lobut, Clémence; Azmat, Ghazala. 2025. "The effect of AI-enhanced instruction on students’ learning, evidence from a randomized controlled trial." AEA RCT Registry. AEARCTR-0015954. https://doi.org/10.1257/rct.15954-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
What is the impact of Artificial Intelligence (AI) use on student learning and teachers’ pedagogical practices? With the rise of AI, a central question arises as to the added value of these technological innovations. In particular, we're interested in the effects of adaptive AI, i.e. AI that enables teachers to personalize the exercises offered to students according to their profile as learners. We randomly assign a large sample of over 300 high schools in France to an active control group and two treatment groups. We seek to study the effects of an AI tool on students' learning (math and french) and socio-emotional skills (collaboration and self efficacy), as well as on teachers’ practices, during the first year of high school. To reduce the risk of attrition and to be able to attribute the expected effects to AI and not to an increase in time spent working on math or French, the control group is active and has access to a version of the application without the AI component. While the first treatment arm has only access to this AI component, the second treatment arm will also have access to extra functionalities, aimed at encouraging collaboration among students.
Intervention
In December 2023, the French Ministry of Education announced the launch of an app in September 2024 for students in grade 10 (first year of high school in France), to support students’ learning in French and Mathematics, as an answer to France's poor results to PISA (Programme for International Student Assessment, led by the OECD).
The app’s primary focus is an adaptive tool based on AI that seeks to adapt the exercises offered to students based on their profiles. Hence, the main goal of this tool is to support French and math teachers in managing the diversity of their classes in terms of skills.
The intervention we study gives access to this app to a large set of high schools. We randomize schools into three groups.
In the control group, students and teachers have access to a simpler version of the app and can access a large library of exercises but teachers have to select the exercises presented to students. Hence, it is comparable to a digital textbook.
In the first treatment group, students and teachers have access to the same catalog of exercises, but also to the AI module that selects the exercises submitted to students, based on an algorithm. This algorithm is based on the theory of the proximal development zone: an exercise should not be too easy, so as not to bore the student, but should not be too difficult, so as not to discourage him or her.
In the second treatment group, students and teachers have access to the large catalog of exercises, to the AI tool, as well as to a tool aimed to promote student collaboration. Teachers can pair students together, so that they have to answer together a set of exercises. In particular, the weakest student proposes an answer, which the stronger pupil must accept or reject. The first student then has the opportunity to correct his or her answer. Hence, is it comparable to peer tutoring.
For all three versions, the scope of the exercises is similar and covers the curriculum in grade 10 in French and Math. The app’s main purpose is to be a remedial tool, helping teachers personalize their instruction to lower achieving students. However, it can be used with all students.
Experimental design
The intervention will be implemented in schools identified as voluntary by regional authorities in charge of digital education. Regional authorities had 3 months to identify all voluntary high schools. Both vocational and general tracks are included in the study. A small pilot took place the year before, and schools are excluded from the study. Private schools are excluded as well, as too few of them were identified as voluntary.
All public voluntary high schools that were not part of the pilot are included in the study. We randomize at the school level. All grade 10 French and math teachers have the possibility to take part in the study. The baseline survey is administered before treatment begins.
Randomization is done at the school level. Schools were randomly assigned to one of the three groups at the beginning of the year. For technical purposes, randomization was done prior to the baseline questionnaire.
Design details
Cited by
No citation data available for this registration yet.