Humans versus Smartbots: Scaling-up behavioral interventions to reduce teacher shortages
How to cite
Ajzenman, Nicolas; Elacqua, Gregory; Hermosilla, Catalina; Nuñez, Graciela Perez; Mijares, Ana Teresa del Toro. 2024. "Humans versus Smartbots: Scaling-up behavioral interventions to reduce teacher shortages." AEA RCT Registry. AEARCTR-0014942. https://doi.org/10.1257/rct.14942-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We design an experiment to assess the relative effectiveness of human mentors versus AI-powered chatbots in motivating students to pursue a teaching career in Chile. We work with Elige Educar, a Chilean NGO that implements a mentoring program for students interested in education or related fields. Participating high school students are randomly assigned to one of three groups: a group that re- ceives support from a human mentor via WhatsApp; a group that receives support from an AI-powered chatbot, also through WhatsApp; and a passive control group that receives no support. We assess the effectiveness of each treatment in motivating students to register for an education major through Chile’s centralized university admissions process. Beyond addressing Chile’s immediate teacher recruitment challenges, the study contributes to the emerging research on AI in education by empirically assessing the potential of generative AI to provide personalized, scalable mentoring.
Intervention
The intervention is a randomized controlled trial comparing two methods of mentoring high school students in Chile about teaching careers: human mentors and an AI-powered chatbot named "Kai." Working with the NGO Elige Educar, the study targets students who registered for Chile's university entrance exam (PAES) and expressed interest in education or social sciences careers.
Students are randomly assigned to receive WhatsApp-based mentoring from either trained human mentors (typically education students), an AI chatbot trained on Elige Educar's resources using ChatGPT-4mini, or no support (control group). The mentoring occurs from October 2024 to January 2025, with weekly activation of student subgroups. Both human mentors and the chatbot provide information about education programs, scholarships, career benefits, and application requirements. They make up to two contact attempts per student during their assigned intervention week.
A key innovation is expanding beyond students already interested in teaching to include those interested in social sciences more broadly. The intervention includes an intensive support phase during university applications in January 2025, when both treatment arms contact all non-opted-out students to provide application guidance. The study will evaluate effects on students' likelihood of applying to education programs through Chile's centralized university admissions process.
See PaP for more details.
Experimental design
The study is a randomized controlled trial evaluating the effectiveness of AI-powered chatbots versus human mentors in motivating Chilean high school students to pursue teaching careers. The experiment has three arms:
Human mentor treatment: Students receive WhatsApp mentoring from trained education students using Elige Educar's "Quiero Ser Profe" methodology.
AI chatbot treatment: Students interact with "Kai," an AI-powered chatbot trained on Elige Educar's resources using ChatGPT-4mini.
Control group: No contact or support.
The sample includes 43,539 students who registered for Chile's university entrance exam and expressed interest in education (54.4%) or social sciences (45.6%). Treatment assignment is stratified by gender, school type (public/private-subsidized/private), and initial career interests. The human mentoring arm includes 9,000 students, chatbot arm 20,000 students, and control group 14,539 students.
Both treatment arms operate October 2024-January 2025, making two contact attempts per student during their assigned week. An intensive support phase occurs during university applications (January 2024). The intervention uses Elige Educar's WhatsApp Business account to ensure source credibility.
Primary outcomes measure students' likelihood of applying to education programs through Chile's centralized university admissions system. Administrative data comes from DEMRE (Department of Evaluation, Measurement and Educational Registry).
Design details
Cited by
No citation data available for this registration yet.