It Takes a Village (Plus AI): A Pilot Study on AI-Driven Math Tutoring to Revolutionize Learning
How to cite
Thomas, Danielle; Koedinger, Kenneth R.. 2024. "It Takes a Village (Plus AI): A Pilot Study on AI-Driven Math Tutoring to Revolutionize Learning." AEA RCT Registry. AEARCTR-0014505. https://doi.org/10.1257/rct.14505-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Personalized Learning Squared (PLUS) is an edtech platform that uses a hybrid human-AI tutoring intervention to boost learning gains for middle school students from historically underserved communities. We know that tutoring works to improve learning, especially among students who are furthest behind. However, tutoring is a fairly resource intensive and expensive intervention. PLUS connects remote, human tutors to students working on math software in classrooms, and uses insights from the math software to identify needs in real time and help tutors allocate their attention between students. This optimizes student gains and provides a scalable solution to increase math learning. PLUS has been operating in classrooms since the 2022-2023 school year. They have been expanding into new schools and districts, improving the PLUS platform as well as their approach to onboarding and supporting teachers and tutors, and evaluating the impact of the PLUS platform iteratively. They plan to conduct a pilot study during the 2024-2025 school year that will continue to test the implementation of the PLUS program as well as the impact on student learning outcomes.
Intervention
PLUS Tutoring is a math tutoring program that operates within school hours during regular math classes and is designed to provide personalized tutoring for all students in a school. Using data from schools’ existing math practice software (Mathia, iReady, IXL, etc.), the PLUS Tutoring software helps allocate appropriate levels of tutoring to different students. Students in PLUS will receive one of three forms of PLUS tutoring: AI/ed tech tutor added (no human tutoring), on-demand student-initiated tutoring, or data-driven tutor-initiated tutoring. All students in the study will, one day per week during math class, use a device (typically a laptop) with educational technology systems, and open a video-conferencing platform (such as Zoom or Pencil Spaces). Treatment students will receive data-driven tutor-initiated tutoring, with tutors assigned to 2-6 students based on student needs and tutoring skills. Tutors will lead sessions between 2 and 20 minutes to build relationships, identify and address motivational challenges and help get students back on track to making progress. Tutors use a dashboard (PLUS Toolkit) to track student math achievement, practice effort, and prior progress. Comparison group students will have access to on-demand student-initiatives tutoring, with 1 tutor available to serve 10-15 students. During math classes, students may use video-conferencing platforms to request help from a remote tutor. These remote tutoring sessions are also predicted to last roughly 2-20 minutes.
Experimental design
We will use a regression discontinuity design (RD) to measure the impact of tutor-initiated versus student-initiated tutoring on math learning outcomes among middle school students. Previous test scores will be used as the forcing variable and a score cutoff will determine allocation into tutor-initiated or student-initiated tutoring. Students whose school records do not include this score will be randomly assigned. Spring 2024 PSSA/WVGSA scores will be used as the forcing variable for the RD. Anyone below the median PSSA score for a particular district, school, and grade, will be assigned to tutor-initiated tutoring, while students who score above the median will be assigned to student-initiated tutoring. Students with missing scores will be randomly assigned to one of the two groups. Additionally, we will analyze differences in outcomes across key subgroups, including historically marginalized students, students from low-income backgrounds, students with disabilities, English Language learners, and students from underserved populations. This will allow us to explore how these targeted groups respond to different tutoring models, shedding light on any variations in impact.
Design details
Cited by
No citation data available for this registration yet.