Measuring Algorithmic Bias in Online Job Recommendation Systems

How to cite

Zhang, Shuo; Kuhn, Peter. 2020. "Measuring Algorithmic Bias in Online Job Recommendation Systems." AEA RCT Registry. AEARCTR-0006101. https://doi.org/10.1257/rct.6101-1.2

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Shuo Zhang* , Peter Kuhn

* Primary investigator

Registry ID
AEARCTR-0006101
DOI
https://doi.org/10.1257/rct.6101-1.2
First registered
Last updated
Status
In development
Trial dates
2020-07-01 – 2024-06-30
Countries
Private
JEL codes
J16, J71, C93
Registry record
http://www.socialscienceregistry.org/trials/6101

Abstract

Virtually all internet job boards now recommend jobs to the workers who use their platforms. These job recommendations are generated by algorithms, using criteria that include the worker’s characteristics and previous search behaviors, and the match between the worker’s characteristics and the job’s requirements. While job recommendation algorithms have the potential to help workers find their preferred jobs faster and to improve worker-job match quality, they also have sparked deep concern about the fairness of the job recommendation algorithms: even when this is not intended by the designers, the recommended jobs may reinforce gender and other stereotypes. We plan to conduct a resume audit study on several leading online job boards in China to learn whether, to what extent, and how job board algorithms systematically treat male and female job seekers differently. Specifically, we will collect the jobs that are recommended to fictitious workers who have identical backgrounds except for gender, compare the job recommendation outcomes and measure the difference of outcomes between two gender groups.

Intervention

We will set up fictitious profiles and send fictitious applicants to jobs posted on online job boards.

Experimental design

Our research design is a type of resume audit study, which begins by creating otherwise identical male and female worker profiles on Chinese job boards, then observing which jobs are recommended to the profiles.

Design details

Randomization method
In our audit study, the resumes come in pairs, in which the two workers in each pair have identical backgrounds except for gender.
Randomization unit
Individual fictitious profile.
Clusters
No cluster
Observations
We plan to create about 140 fictitious profiles on each job board and replicate these profiles and application processes across a few big cities. Each profile will apply to a substantial number of jobs, which are recommended to profile by the applicant. .
Arms
Half of the fictitious profiles are females, and half of the fictitious profiles are males.
Primary outcomes
Characteristics of jobs recommended by online job boards.
Secondary outcomes
Reactions received by the fictitious resumes.

Cited by

No citation data available for this registration yet.

Keywords