Does large language model technology really equalize low- and high-skill workers?

How to cite

Lombardi, Maria; Cruces, Guillermo; Galiani, Sebastián; Galvez, Ramiro. 2025. "Does large language model technology really equalize low- and high-skill workers?." AEA RCT Registry. AEARCTR-0016607. https://doi.org/10.1257/rct.16607-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Maria Lombardi* , Guillermo Cruces , Sebastián Galiani , Ramiro Galvez

* Primary investigator

Registry ID
AEARCTR-0016607
DOI
https://doi.org/10.1257/rct.16607-1.1
First registered
Last updated
Status
In development
Trial dates
2025-09-01 – 2026-02-15
Countries
Private
JEL codes
J24, O33
Registry record
http://www.socialscienceregistry.org/trials/16607

Abstract

This study investigates the impact of large language model (LLM)-based chatbots on the labor market, focusing on their potential to reduce performance gaps between low- and high-skilled workers. Our experiment recruits participants with diverse skill levels, defined by educational attainment, and evaluates their performance in a task. The treatment group will complete this task with the assistance of an LLM-based chatbot, while the control group will complete it without any assistance. We will examine the differences between high- and low-skilled individuals in terms of how LLM assistance affects performance in the task.

Experimental design

We will conduct an online experiment in which respondents will be presented with an incentivized task based on a realistic business scenario. Participants in the treatment group will have access to an LLM-based chatbot, whereas participants in the control group will not. After completing the main task, all participants, regardless of their assigned group, will answer a series of follow-up questions related to this task, without access to the LLM-based chatbot.

Design details

Randomization method
After providing some demographic characteristics, respondents will be randomly assigned to the treatment and control groups. The randomization will occur at the individual level and will be implemented by the survey software (Qualtrics).
Randomization unit
Randomization is conducted at the individual level
Clusters
Treatment is not clustered
Observations
We will aim for a sample size of N=600. However, there are logistical challenges and costs associated with gathering the necessary sample. One of the main constraints is the need to perform the task using a computer. If the sample gathering goes smoother than expected, we will gather an even greater sample size.
Arms
Half of the participants will be assigned to the treatment group (n=300), and the other half will be assigned to the control group (n=300).
Primary outcomes
Performance and time taken to complete the main task, and performance on the follow up questions.
Secondary outcomes
Perceptions of task difficulty

Cited by

No citation data available for this registration yet.

Keywords