AI Tutor and Student Outcomes
How to cite
Khoury, Stephanie El; Aucejo, Esteban; Vina, Patricia Ramirez De La; Peyre, Agustina Affonso. 2025. "AI Tutor and Student Outcomes." AEA RCT Registry. AEARCTR-0016493. https://doi.org/10.1257/rct.16493-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We evaluate the impact of AI-powered instructional bots on student learning in introductory microeconomics courses at a public university. These conversational bots are designed to teach students core concepts by answering questions, walking them through practice problems, and providing tailored explanations in real time. Each bot is aligned with a specific topic(such as demand, supply, market equilibrium, and elasticity) and is embedded into weekly course modules. The randomized controlled trial assigns students to receive access to the bots for certain topics, while others follow the standard instructional materials alone. We examine effects on students’ course outcomes. The study sheds light on whether AI tools can serve as scalable instructional supplements in economics education, particularly for students who may need additional support outside of class.
Intervention
Students are randomly assigned to receive access to interactive AI bots designed to teach core microeconomics topics such as demand, supply, and elasticity. The bots provide personalized feedback, topic-specific guidance, and reflective prompts via ASU’s CreateAI platform. Students in the control group receive standard instructional materials only. The intervention is conducted over five weeks.
Students assigned to the treatment group receive a total of five instructional emails, each aligned with a major topic in the course (e.g., demand, supply, elasticity). Each message introduces the relevant AI bot for that topic and includes a guide on how to interact with it via the ASU CreateAI Builder platform. These bots function as conversational tutors, capable of walking students through course content, problem-solving strategies, and real-time feedback. The first Bot is tied to a required course assignment implemented via Canvas Quizzes, which prompt structured interactions with the bots through open-ended questions to help treated students get used interacting with the bot. All students complete this graded assignment . The treatment group uses the bot to complete the assignment; the control group uses regular materials.
In addition to the instructional emails, students in the treatment group also receive bot reminders before major assessments (such as midterms). These reminders encourage students to revisit the bots and review key content.
Students in the control group receive parallel weekly emails that serve as neutral study reminders, directing them to review course material posted on Canvas. Before major assessments, they receive reminders that mirror the treatment group's message in structure and tone but exclude mention of the bots.
All students complete two Qualtrics surveys:
- Baseline Survey: Administered in the first weeks of class, collects demographic background, study routines, work status, AI familiarity, and expectations for the course. This data is also used to stratify the randomization. Randomization is at the average high school GPA level.
- Follow-Up Survey: Administered in the final weeks of the semester. This survey measures self-reported learning, confidence, engagement, perceptions of the bots (for treated students), and other academic behaviors. Both surveys are incentivized with extra credit.
Experimental design
Students enrolled in four introductory microeconomics courses at ASU in Fall 2025 are invited to participate in the study via course announcements and Canvas messages. Following consent and completion of a baseline survey, students are randomized at the individual level using a stratified procedure. Approximately half of the students are assigned to receive access to instructional AI bots aligned with five course topics. The other half receive standard instructional materials only.
Treatment is at the individual level and delivered over five weeks. All students complete a graded assignment aligned with the first bot topic. The treatment group uses the bot to complete the assignment; the control group uses regular materials. Assignments are graded for completion, not correctness, and receive post-assignment feedback through Canvas announcements. Surveys and extra credit incentives are offered to encourage participation. The stratification is at the level of the average high school gpa.
Treatment group students receive five instructional messages (one per bot/topic) and exam reminder emails encouraging bot use. These messages include how-to guides, usage tips, and links to the CreateAI Builder platform.
At the end of the semester, all students are invited to complete a second survey, which captures self-reported learning outcomes, usage of instructional tools, engagement patterns, and (for treated students) evaluation of the bots.
Design details
Cited by
No citation data available for this registration yet.