Experimental Study on the Impact of Generative Artificial Intelligence on the Quality of Individual Career Decision-making

How to cite

wang, xinyue. 2026. "Experimental Study on the Impact of Generative Artificial Intelligence on the Quality of Individual Career Decision-making." AEA RCT Registry. AEARCTR-0017680. https://doi.org/10.1257/rct.17680-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

xinyue wang*

* Primary investigator

Registry ID
AEARCTR-0017680
DOI
https://doi.org/10.1257/rct.17680-1.0
First registered
Last updated
Status
In development
Trial dates
2025-11-03 – 2026-03-14
Countries
China (Shenzhen, Guangdong Province≠)
JEL codes
J24, D91, O33
Secondary IDs
N/A
Registry record
http://www.socialscienceregistry.org/trials/17680

Abstract

This randomized controlled experiment studies how generative artificial intelligence affects college students' career decision-making. Participants were randomly assigned to rank job opportunities with or without the assistance of generative AI. The quality of decision-making is objectively measured by comparing the rankings of participants with the benchmarks derived from the BERT-based semantic matching model. This study tested whether artificial intelligence has improved the accuracy of job matching, enhanced decision-making confidence, and whether these effects vary depending on individual characteristics (such as gender, education level, major) or trust in artificial intelligence. The research results aim to provide information for the application of artificial intelligence in career counseling and offer objective indicators for evaluating the quality of decision-making.

Intervention

Participants are randomly assigned to either an AI-assisted decision-making condition or a control condition. In the AI-assisted condition, participants are allowed to consult a generative artificial intelligence tool when completing a simulated career decision task. In the control condition, participants complete the same task without access to any external decision-support tools. Aside from the availability of AI assistance, all other experimental procedures are identical across groups.

Experimental design

The study employs a randomized controlled experimental design. Eligible participants are randomly assigned at the individual level to either an AI-assisted treatment group or a control group. All participants complete the same simulated career decision task, with the only difference being access to a generative AI tool in the treatment condition. Outcomes are measured immediately after task completion.

Design details

Randomization method
Randomization is conducted by a computer using a pre-generated random assignment algorithm. Participants are assigned to treatment or control groups at the individual level, with stratification by key baseline characteristics.
Randomization unit
Randomization is performed at the individual level. Prior to randomization, participants are stratified by gender (Sex), education level (Education), and academic major (Major) to ensure balanced distribution of these characteristics across the control and treatment groups. Within each stratum, participants are then randomly assigned to either the control group (no AI assistance) or the treatment group (AI-assisted).
Clusters
Not applicable (non-clustered design).
Observations
180 to 240 individual participants.
Arms
Control group: 60 to 120 individuals; Treatment group (AI-assisted): 60 to 120 individuals.
Primary outcomes
Career decision quality, measured as the rank-order consistency between participants’ job-offer rankings and an algorithm-generated benchmark ranking.
Secondary outcomes
Secondary outcomes include: (1) alternative measures of ranking accuracy such as Spearman’s rho and top-k overlap rates; (2) self-reported decision confidence; (3) perceived decision difficulty; and (4) adoption of AI advice in the treatment group.
Sponsors
Name: School of Economics and Management, Harbin Institute of Technology (Shenzhen) Location: Shenzhen, Guangdong Province, China Url: https://www.hitsz.edu.cn/

Cited by

No citation data available for this registration yet.

Keywords