Pre-Registration: Understanding the impact of AI on the appeal of factory jobs

How to cite

Molak, Ekaterina; Hesler, Nikolaus von; Sadun, Raffaella; Englmaier, Florian; Tamayo, Jorge. 2025. "Pre-Registration: Understanding the impact of AI on the appeal of factory jobs." AEA RCT Registry. AEARCTR-0016757. https://doi.org/10.1257/rct.16757-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ekaterina Molak* , Nikolaus von Hesler , Raffaella Sadun , Florian Englmaier , Jorge Tamayo

* Primary investigator

Registry ID
AEARCTR-0016757
DOI
https://doi.org/10.1257/rct.16757-1.1
First registered
Last updated
Status
On going
Trial dates
2025-04-01 – 2026-02-28
Countries
Private
JEL codes
C93, J23, J24, M51, O33
Registry record
http://www.socialscienceregistry.org/trials/16757

Abstract

This study examines how referencing artificial intelligence (AI) in job postings affects job seeker behavior in industrial labor markets. In a randomized field experiment, Siemens will post job advertisements for factory roles either with or without mentions of AI, rotating the treatment condition weekly at the job-post level. The ads will appear on the Siemens Career Portal and be syndicated across major recruitment platforms (e.g., LinkedIn, Indeed). The sample includes both blue-collar (e.g., shopfloor operators) and white-collar (e.g., industrial engineers) positions across multiple Siemens factories in countries such as the United States and China. The study will assess differences in application volume and applicant characteristics across conditions. Data will be collected during the summer-autumn period of 2025. The findings will shed light on how AI-related messaging influences job seekers' interest and selection into jobs, informing both economic understanding of labor market perceptions and practical recruitment strategies amid growing AI adoption.

Intervention

1. Mentions of AI in Job Postings
The intervention involves systematically varying whether artificial intelligence (AI) is referenced in job advertisements for factory roles. Each job posting will be alternately published in one of two formats:

1.1.AI Condition: The job description includes references to AI in both the title and the main body
- Title: XYZ in the factory with AI-powered machines
- Body: Here you will work in a modern, dynamic factory environment, where we increasingly rely on artificial intelligence (AI) to optimize production processes and drive innovation

1.2.Non-AI Condition: The job description omits any mention of AI in both the title and the main body
- Title: XYZ
- Body: Here you will work in a modern, dynamic factory environment.

2. Randomization and Rotation Schedule
The assignment of AI mentions is randomized at the job-post level. For each role, the job advertisement will be published in one condition (AI-framed or non-AI) for a period of one week after which the alternative version will be posted. This within-job, cross-time randomization strategy ensures that candidates are exposed to only one version of the job ad at any given time, minimizing cross-condition contamination. The rotation schedule is designed to allow within-role comparisons over time while maintaining ecological validity in a live recruitment environment.

Labor split: Siemens is responsible for designing the experiment and will handle the randomization, scheduling, and execution. The research team (Jorge Tamayo, Raffaella Sadun, Florian Englmaier) provided input on the experimental design (e.g., rotation logic), but they will not oversee the experiment's design or implementation. The research team will analyze the resulting data to evaluate the intervention's impact.

Experimental design

A field experiment will be conducted to measure the impact of referencing artificial intelligence (AI) in job postings on candidate behavior. The intervention will be implemented across multiple Siemens factories in different countries (e.g., the U.S. and China), using live recruitment postings for both blue-collar (e.g., machine operators) and white-collar (e.g., industrial engineers) factory roles.

Each eligible job posting will be randomly assigned to start in either the AI or non-AI condition, and will alternate weekly between the two formats. Each post is expected to undergo at least one full rotation cycle—i.e., it will be live for one week in the AI condition and one week in the non-AI condition—with some postings possibly undergoing multiple cycles depending on the length of the recruitment period.

Design details

Randomization method
Randomization Method Random assignment of AI mention conditions is implemented using a structured rotation protocol at the job-post level. Within each location, every second eligible job posting is randomly assigned to start in the AI condition, while the others begin in the non-AI condition. Following this initial assignment, each job post alternates weekly between the two conditions (AI vs. non-AI) for the duration of the posting. This allows for within-role comparisons over time while preserving ecological validity. The assignment sequence is automated and tracked using a centralized system maintained by Siemens' recruitment operations team. Treatment status is not visible to recruiters or applicants, minimizing implementation bias. Random assignment of AI mentions conditions is implemented using a structured alternation protocol. Every second job posting within each location is initially published with AI mentions, while the others begin in the non-AI condition. The assignment sequence is generated and tracked using a computerized system and is concealed from recruiters and candidates to prevent bias.
Randomization unit
The unit of randomization is the job-post level, stratified by geography and role type. Randomization occurs independently within each geography (e.g., United States, China) and role category (blue-collar vs. white-collar), resulting in four clusters. Within each cluster, the probability of assignment to either condition (AI-framed or non-AI) is equal.
Clusters
We are randomizing four clusters of job postings.
Observations
We are randomizing between 30-35 job posts, with an expected distribution of approximately 80% blue-collar and 20% white-collar roles – depending on the recruitment needs per a specific location during the study period (June-December 2025)
Arms
We are randomizing between 30-35 job posts, with 50/50 allocation between starting with AI vs non-AI mentioning wording.
Primary outcomes
Primary Outcomes (end points) • Application volume: Total number of applications received per job posting per condition. • Gender ratio: Proportion of male and female applicants per condition. • Hiring speed: Time from job posting to candidate hire
Secondary outcomes
Secondary Outcomes (end points) • AI familiarity: Indicators of prior AI-related knowledge or interest (e.g., keywords in CVs and self-reported skills). • Education level: Highest degree or qualification listed by applicants. • Shortlisting likelihood: Probability that an applicant is invited for an interview. • Hiring likelihood: Probability that an applicant is hired.

Cited by

No citation data available for this registration yet.

Keywords