The Impact of Generative AI on Employee Performance: A Field Experiment in a Multinational Education Technology Company

How to cite

Wu, Hugh Xiaolong; Xu, Huayu. 2025. "The Impact of Generative AI on Employee Performance: A Field Experiment in a Multinational Education Technology Company." AEA RCT Registry. AEARCTR-0015560. https://doi.org/10.1257/rct.15560-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Hugh Xiaolong Wu* , Huayu Xu

* Primary investigator

Registry ID
AEARCTR-0015560
DOI
https://doi.org/10.1257/rct.15560-1.0
First registered
Last updated
Status
In development
Trial dates
2025-03-01 – 2025-08-01
Countries
Private
JEL codes
J24, L80, M12, M50
Registry record
http://www.socialscienceregistry.org/trials/15560

Abstract

This study examines whether promoting the use of Generative AI can improve job performance among course counselors (professional sales) working at a leading education technology company in China, and seeks to understand the underlying reasons for any improvements in job productivity. Approximately 500 counselors will participate in the study. All participants will have access to an internal database powered by ChatGPT, but about one-third (the treatment group) will receive additional encouragement to actively integrate Generative AI into their everyday tasks. The remaining counselors (the control group) will not receive this extra support.

By comparing job performance between the two groups, the study aims to determine if counselors in the treatment group achieve better productivity outcomes—particularly by reducing student attrition between course modules. Specifically, after completing the current module, parents decide whether to enroll their children in the next one. The study assesses whether counselors supported by Generative AI are more effective at encouraging parents to continue enrolling their children, thereby lowering dropout rates. The research will further explore the mechanisms driving any observed improvements in performance.

Intervention

All participating counselors will have access to an internal Generative AI database powered by ChatGPT. Counselors assigned to the treatment group (approximately one-third of participants) will receive additional encouragement to actively integrate Generative AI into their daily work activities. This support includes demonstrations, regular reminders, and resources on effectively using Generative AI tools to enhance interactions with students' parents, aiming to improve their job performance and reduce student attrition between course modules. The control group counselors will receive access to the same Generative AI resources but without additional promotion.

Experimental design

This study uses a randomized controlled trial (RCT) design. Approximately 500 course counselors from a leading education technology company in China will be randomly assigned to either a treatment or a control group. All counselors will have access to an internal Generative AI database powered by ChatGPT. However, counselors in the treatment group (about one-third of participants) will receive additional encouragement designed to actively promote the integration of Generative AI into their daily work routines. The remaining counselors (control group) will have access to the same resources but will not receive any additional promotion. The primary outcome—student attrition rates after the current course module—will be compared between the two groups to evaluate the effectiveness of this intervention.

Design details

Randomization method
Randomization will be done by the company's data science team using a random number generator.
Randomization unit
The randomization will be done at the counselor group level, with each group consisting of approximately 12 counselors.
Clusters
~ 40 counselor groups
Observations
~ 500 counselors
Arms
Approximately one-third of the counselor groups were assigned to the treatment condition, while two-thirds were assigned to the control group.
Primary outcomes
customer attrition

Cited by

No citation data available for this registration yet.

Keywords