Experimental Study on the Impact of Generative Artificial Intelligence on Individual Career Preferences

How to cite

wang, xinyue. 2026. "Experimental Study on the Impact of Generative Artificial Intelligence on Individual Career Preferences." AEA RCT Registry. AEARCTR-0017719. https://doi.org/10.1257/rct.17719-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

xinyue wang*

* Primary investigator

Registry ID
AEARCTR-0017719
DOI
https://doi.org/10.1257/rct.17719-1.0
First registered
Last updated
Status
Completed
Trial dates
2026-03-21 – 2026-03-29
Countries
China (Shenzhen, Guangdong Province≠)
JEL codes
J24, D91, O33
Secondary IDs
N/A
Registry record
http://www.socialscienceregistry.org/trials/17719

Abstract

This study investigates whether and how generative artificial intelligence (AI) influences individuals’ career choice preferences. Using a 2 (AI intervention: with vs. without) × 3 (decision context: subjective ideal, expert guess, peer guess) mixed factorial experimental design, we randomly assigned 93 university students to either a control group or an AI intervention group. Participants role-played as job seekers and rated six jobs across three decision contexts. Outcomes include skill-matched job ratings, rankings, and top-1 choice probabilities. We hypothesize that generative AI increases preference for skill-matched jobs across all three contexts and that job familiarity moderates this effect. Primary analyses include independent t-tests, linear mixed models, logistic regression, and moderation analysis with simple slopes.

Intervention

The intervention consists of providing static, pre-generated Artificial Intelligence (AI) advice during a simulated career decision-making task. Participants role-play as anonymous job seekers and are randomly assigned to one of two conditions:
1. Control Group (Independent Evaluation): Participants review the personal statements of anonymous job seekers—including education, internships, skills, and hobbies—and independently rate the person-job fit for six distinct job offers on a 1-to-10 scale. No external assistance or reference information is provided.
2. Treatment Group (AI-Assisted Evaluation): Participants complete the same evaluation task but receive an exogenous information shock: static AI-generated advice is displayed alongside each job offer. The advice, pre-generated by a large language model acting as a “senior career development consultant,” offers tailored analyses of the matching degree, highlighting professional alignments and identifying potential mismatches (e.g., distinguishing core competencies from hobby-based interests).
The AI advice is presented statically to eliminate confounding factors related to participants’ prompt-engineering skills or variation in human–AI interaction frequency, ensuring uniform treatment administration across all participants in the treatment group.

Experimental design

This study employs a 2 × 3 mixed factorial experimental design. The between-subjects factor is AI intervention (control vs. treatment), and the within-subjects factor is decision context (subjective ideal, expert guess, peer guess).
Participants are randomly assigned to either the control group (no AI advice) or the treatment group (AI advice provided). All participants complete three decision-making phases in a fixed order: (1) subjective ideal context, (2) expert guess context, and (3) peer guess context. In each phase, participants evaluate six job offers on a 1-to-10 scale and rank them by preference.
The primary outcomes are: (a) average rating for skill-matched jobs, (b) rank of skill-matched jobs, and (c) an indicator for whether a skill-matched job is selected as the top choice. Data are collected via an online platform. The study is pre-registered prior to data analysis.

Design details

Randomization method
Computer-generated randomization. The randomization was implemented automatically upon participant enrollment and before any experimental tasks were presented.
Randomization unit
The randomization was conducted at the individual participant level. Each participant was independently assigned to either the control group or the treatment group. No clustering or group-level randomization was employed, as the experimental design involves a between-subjects manipulation of AI intervention with individual-level assignment.
Clusters
Not applicable (non-clustered design).
Observations
100 to 120 individual participants.
Arms
Control group: 50 to 60 individuals; Treatment group (AI-advice): 50 to 60 individuals.
Primary outcomes
1. Skill-matched job rating: The average rating (on a 1-to-10 scale) assigned to three pre-defined skill-matched jobs. Skill-matched jobs are identified based on alignment between the job seeker’s profile and job requirements, as validated by the AI advice and expert judgment. 2. Skill-matched job rank: The rank order of skill-matched jobs among the six job offers, with lower ranks indicating higher preference (rank 1 = most preferred). This is derived from the sorting task in each experimental phase. 3. Top-1 choice indicator: A binary variable equal to 1 if the participant selects a skill-matched job as their most preferred option in the subjective ideal context, and 0 otherwise. These outcomes are measured separately for each of the three decision contexts: subjective ideal, expert guess, and peer guess.
Secondary outcomes
1. Decision confidence: Measured on a 7-point Likert scale (1 = not at all confident, 7 = very confident) assessing participants’ confidence in their person-job fit ratings and final rankings. 2. Perceived task difficulty: Measured on a 7-point Likert scale (1 = very easy, 7 = very difficult) capturing participants’ subjective assessment of the complexity of the evaluation task. 3. AI trust and adoption (treatment group only): Three items measured on 7-point Likert scales assessing (a) trust in AI advice, (b) use of AI advice as a reference, and (c) perceived influence of AI advice on final ratings. 4. Effort: Measured on a 7-point Likert scale (1 = strongly disagree, 7 = strongly agree) capturing self-reported effort investment in the ranking task. 5. Rationality of decision: Measured on a 7-point Likert scale assessing participants’ perception that their final ratings and rankings constitute a reasonable career decision.

Cited by

No citation data available for this registration yet.

Keywords