Adoption and Effectiveness of AI Tutoring in Tertiary Education
How to cite
Hermes, Henning; Isphording, Ingo. 2024. "Adoption and Effectiveness of AI Tutoring in Tertiary Education." AEA RCT Registry. AEARCTR-0014959. https://doi.org/10.1257/rct.14959-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We study whether and how much AI technologies improve learning and educational outcomes in a tertiary education context. We test this question using a large-scale randomized field experiment. Adapting an encouragement design, we evaluate which students select into usage of a targeted AI assistant within an online educational program, and assess the effects of this AI assistant on educational outcomes using data on study progress and grades based on admin data from the university.
Intervention
Our RCT investigates the effects of encouraging students to use an AI assistant on their study success. The AI assistant, integrated into the university's online learning platform, is a Chatbot that offers students support by answering clarifying questions, providing examples, looking up definitions, and referencing relevant study materials. The Chatbot, which is technically built on the current GPT-4 framework, has been trained with direct references to relevant sources such as study letters, further literature or presentations, in order to assist students in their individual learning process. Students in the intervention group will receive specific encouragement to use the Chatbot more intensively, while a control group will not receive such encouragement.
Experimental design
We will conduct a field experiment in which participants are randomly assigned to either the treatment or control group. Participants in the treatment group will receive encouragement to use an AI assistant (Chatbot), while those in the control group will not receive this encouragement. The AI assistant is generally available for all students. All students will be part of the experiment, either in the treatment group or the control group. The experiment will also include a baseline survey for all students to gather information on their perceptions and usage of the Chatbot, as well as personal characteristics. Main outcomes will be provided with administrative data from the university as well as an endline survey.
Design details
Cited by
No citation data available for this registration yet.