NUMI, Mastery Learning, and Short-Duration Math Practice

How to cite

Oreopoulos, Philip; Liut, Michael; Sangu, Alp. 2026. "NUMI, Mastery Learning, and Short-Duration Math Practice." AEA RCT Registry. AEARCTR-0018315. https://doi.org/10.1257/rct.18315-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Philip Oreopoulos* , Michael Liut , Alp Sangu

* Primary investigator

Registry ID
AEARCTR-0018315
DOI
https://doi.org/10.1257/rct.18315-1.0
First registered
Last updated
Status
Completed
Trial dates
2026-03-30 – 2026-04-03
Countries
United States of America (Chattanooga)
JEL codes
I21, I28, C93, O33
Registry record
http://www.socialscienceregistry.org/trials/18315

Abstract

We are studying whether short classroom math practice activities can be improved by combining mastery learning with an AI tutor. Middle school students are randomly assigned to different versions of an online math practice activity during class. Some students receive access to an AI tutor, while others do not. Some students complete the activity under mastery-learning rules, meaning they must watch enough of the lesson video and answer questions correctly before moving on, while others can move through the activity more freely. Students are also randomly assigned which math topic they practice.

About one week later, students take a test that includes questions on both the topic they practiced and a topic they did not practice. This allows us to compare how much students learned from the activity and whether the effects differ depending on AI access and mastery learning. We also study how far students progress through the activity, how they interact with the AI tutor, and whether technical issues such as website speed may affect results.

Intervention

Students participated in an online classroom math practice activity during a regular class period. Students were randomly assigned to receive access to an AI math tutor or not, and to complete the activity either under mastery-learning rules or under a more flexible non-mastery format. Students were also randomly assigned which of two math topics to practice. In the mastery condition, students had to watch at least part of the instructional video and answer questions correctly before moving on. In the non-mastery condition, students could move through the material more freely. A week later, students completed a follow-up test.

Experimental design

This study uses randomized assignment of middle school students to different versions of an in-class online math practice activity. Students are randomly assigned to: (i) access to an AI tutor or not, (ii) mastery-learning rules or not, and (iii) one of two math topics to practice. About one week later, students complete a follow-up test covering both the topic they practiced and the topic they did not practice. The main analysis compares, within student, performance on practiced versus unpracticed material across treatment groups.

Design details

Randomization method
Randomization was done by computer through the online platform at the student level.
Randomization unit
Individual student. Topic assignment, AI assignment, and mastery assignment were all randomized at the individual student level.
Clusters
Not applicable; randomization is at the individual student level. If the registry insists on an entry, use: 0 clusters (individual-level randomization) or N/A - individual-level randomization, depending on what it accepts.
Observations
Approximately 8000 students in grades 6–8
Arms
50 50 split
Primary outcomes
The primary outcome is student performance on the week-later test, measured as the within-student difference between performance on questions corresponding to the topic assigned for practice and performance on questions corresponding to the topic not assigned for practice.
Secondary outcomes
Secondary outcomes include: total follow-up test score score on the practiced-topic questions score on the unpracticed-topic questions exit ticket score progression through the practice activity, including whether students reached the second set and the exit ticket mastery-specific progression outcomes, including whether students cleared the first and second mastery gates AI usage and engagement measures, including whether students used the AI tutor, number of interactions, and time spent interacting with the tutor

Cited by

No citation data available for this registration yet.

Keywords