The Effects of Generative AI on Learning
How to cite
Irmert, Natalie; Franco, Catalina; Isaksson, Siri. 2024. "The Effects of Generative AI on Learning." AEA RCT Registry. AEARCTR-0014930. https://doi.org/10.1257/rct.14930-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Artificial Intelligence (AI) is becoming an increasingly important skill in the labor market, but will it also affect academic success? Recent research shows that current students --who will be facing this rapidly changing labor market-- are adopting AI tools at differential rates based on both gender and ability. Whether AI will affect adopters’ academic success hinges on whether AI interferes with or enhances learning, which in turn depends on whether AI is being used as a substitute for or complement of own effort. If AI harms learning, students with high adoption rates would be worse prepared for the labor market than those with low adoption rates. If AI enhances learning, students who do not become proficient at AI would be left behind. To assess the impact of AI on learning, we run a controlled lab experiment which allows us to restrict and allow access to AI in different between-subject treatment variations in which students learn about a new topic. We explore whether AI is employed in a way that causally creates a gap in learning and as a consequence payoffs. Our results provide evidence on the important question of whether the documented differential adoption and use of AI by gender and ability is likely to create gender gaps in academic success. In addition, we explore several mechanisms such as over-reliance on generative AI tools, engagement and motivation.
Intervention
We design a laboratory experiment with three between subject treatment variations. In the baseline treatment, students learn without access to AI. In the AI-assisted treatment, students learn with access to AI. In the AI-guided treatment, students learn with access to AI and examples of ways to interact with AI to promote learning. This will allow us to answer the crucial question of whether, and for whom, AI is helpful as a learning tool and whether providing guidance of how to use the tool matters.
Experimental design
Regardless of treatment, subjects will learn about the same topic. The treatment differences lies in whether or not they have access to ChatGPT/ChatGPT with guidance while learning.
Each session will adhere to the same procedure:
-Subjects learn for 15 minutes using written learning materials provided by the researchers.
-Subjects complete practice questions with access to different learning aids depending on treatment.
-Subjects take a 10 minute test without access to any learning aids.
-Subjects take a post-study survey asking them about demographics as well as attitudes and beliefs about AI.
Design details
Cited by
No citation data available for this registration yet.