Teaching Econometrics with AI: Evidence from a Randomized Trial

How to cite

Gallegos, Sebastian. 2025. "Teaching Econometrics with AI: Evidence from a Randomized Trial." AEA RCT Registry. AEARCTR-0016770. https://doi.org/10.1257/rct.16770-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sebastian Gallegos*

* Primary investigator

Registry ID
AEARCTR-0016770
DOI
https://doi.org/10.1257/rct.16770-1.1
First registered
Last updated
Status
Completed
Trial dates
2025-09-01 – 2025-12-23
Countries
Chile ()
Registry record
http://www.socialscienceregistry.org/trials/16770

Abstract

This study evaluates the impact of promoting the use of GPT EconometríaUAI, a conversational AI assistant, on undergraduate students’ learning outcomes in econometrics. The intervention will be implemented during the second semester of 2025 at Universidad Adolfo Ibáñez across seven sections of the Econometrics course. Students will be individually randomized within sections into treatment and control groups. The treatment group will receive a series of emails encouraging the use of GPT EconometríaUAI for study support, while the control group will not receive such promotion (placebo emails may be used).
The primary outcome is student achievement on the first midterm exam. Secondary outcomes include satisfaction with the course and students’ perceptions of their learning and study experience. The study will provide experimental evidence on the role of AI-based study assistants in shaping both academic performance and broader student experiences in higher education.

After the midterm intervention, I implemented an additional intervention for the final exam. This second intervention targeted how students used the GPT rather than whether they used it. Students assigned to treatment received three emails prior to the final exam encouraging learning-oriented use of the tool. The messages promoted tutor-style interaction with step-by-step reasoning, provided a structured prompt to generate practice questions and feedback, and encouraged verification of answers. Control students did not receive guidance emails but retained full access to the GPT, as in RCT 1. To measure compliance and spillovers, the final exam survey asked whether students received the emails and whether they shared the advice.
I implemented a second randomization prior to the final exam using the same sample used for RCT 1. This randomization was independent of RCT 1 and followed the same stratification scheme based on section, gender, and high school type. Within each stratum, half of the students were assigned to treatment and half to control. Similar to the first intervention, the primary outcome is student achievement on the final exam, and secondary outcomes include students’ use and perceptions of their learning and study experience.

Intervention

The intervention consists of promoting the use of GPT EconometríaUAI through emails sent from the official course coordination to students randomized into the treatment group. We plan to send three emails as follows:
Email 1 — Monday at noon (2 days before the exam)
Subject: Have you tried the EconometríaUAI GPT?
Dear [NAME],
Remember that you have access to GPT EconometríaUAI, a tool specially trained with course materials to help you study.
👉 Try it here: [GPT LINK] https://chatgpt.com/g/g-ftvZAGEWO-econometriauai
Explore it today and get better prepared for Wednesday’s exam!
Sincerely,
Econometrics Coordination

Email 2 — Tuesday 11 am (1 day before the exam)
Subject: Questions about Midterm 1? GPT can help you
Dear [NAME],
It’s normal to have questions the day before an exam! GPT EconometríaUAI can support you.
Take advantage of this tool to review your knowledge!
For example, ask:
❓ How do I interpret the constant in a simple regression?
🤖 GPT responds: The constant (intercept) in a simple linear regression represents the predicted value of the dependent variable when the explanatory variable equals zero.
👉 Try it now with your own questions: [GPT LINK] https://chatgpt.com/g/g-ftvZAGEWO-econometriauai
Sincerely,
Econometrics Coordination

Email 3 — Wednesday 10 am (morning of the exam day)
Subject: Last questions before today’s exam?
Dear [NAME],
Today at 6:00 pm is Midterm 1.
Do you have last-minute questions? Watch this short video (<1 minute) and learn how GPT can help you review:
🎥 Watch video: https://www.youtube.com/watch?v=UNHFs4TKi5M
👉 Access GPT here: https://chatgpt.com/g/g-ftvZAGEWO-econometriauai
Best of luck!
Econometrics Coordination

Experimental design

Random assignment will be conducted at the individual level, within each course section, with a 33%-67% split between treatment and control. This assignment will be stratified by observable characteristics available in administrative records (such as gender and high school type), to improve the precision of estimates.

Design details

Randomization method
randomization done in office by a computer, replicable, with stata code
Randomization unit
individual level, within classrooms
Clusters
no clusters
Observations
385
Arms
127 to treatment, 258 to control
Primary outcomes
Grade and total Score on the first econometrics midterm exam. This is the main measure of student academic achievement that the intervention seeks to influence. We will also decompose the score on specific questions, as we expect the GPT to have larger effects on different types of assessments
Secondary outcomes
Course satisfaction. Student self-reported satisfaction with the econometrics course, collected via a post-exam survey. Learning perceptions. Student perceptions of their own understanding and mastery of course content, measured through survey questions. Exposure and use of GPT EconometríaUAI. Self-reported frequency, type of use, and perceived usefulness of the GPT assistant, also collected through the post-exam survey. Exploratory outcomes: Spillovers across peers. Indicators of whether control group students indirectly learned about or used the GPT through classmates, as captured by survey questions on peer communication and study group dynamics. Longer-term academic performance. Scores on subsequent exams (e.g., second midterm, final exam), if the intervention is extended.

Cited by

No citation data available for this registration yet.

Keywords