Going Beyond the Mean: How Gender Affects the Distribution of Evaluations and How the Distribution of Evaluations Affects Hiring Decisions

How to cite

Avery, Mallory; Leibbrandt, Andreas; Vecci, Joseph. 2023. "Going Beyond the Mean: How Gender Affects the Distribution of Evaluations and How the Distribution of Evaluations Affects Hiring Decisions." AEA RCT Registry. AEARCTR-0011877. https://doi.org/10.1257/rct.11877-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Mallory Avery* , Andreas Leibbrandt , Joseph Vecci

* Primary investigator

Registry ID
AEARCTR-0011877
DOI
https://doi.org/10.1257/rct.11877-1.0
First registered
Last updated
Status
In development
Trial dates
2023-08-15 – 2024-08-15
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/11877

Abstract

In this project, we investigate the impact of various distributional metrics of job applicants on the evaluation of applicants and their likelihood of being selected for hiring. We also study whether these distributional metrics have similar effects depending on the gender of the applicants and whether the gender of the applicants is known.

Intervention

In this project, we conduct multiple interventions to investigate the impact of various distributional metrics of job applicants on the evaluation of applicants and their likelihood of being selected for hiring. We also study whether these distributional metrics have similar effects depending on the gender of the applicants and whether the gender of the applicants is known.

Experimental design

Our design aims to measure the impact of the distribution of applicants’ evaluations when making hiring decisions.

Design details

Randomization method
Randomization will be carried out by a computer.
Randomization unit
There are multiple forms of randomization: 1) Randomization into the Gender-Known and No-Gender treatments will be across subjects. Thus, this will be at the individual level. 2) We randomize the pairs shown to each evaluator. That is what pair is shown to each evaluator take place at the individual evaluator level. 3) For each evaluator we also randomize the set of evaluation scores shown. While the evaluation sets are pre-determined what evaluation set is shown to each individual is randomized at the individual evaluator level.
Clusters
See above, in most cases randomization occurs at the individual level, as such we cluster at the individual level.
Observations
Each evaluator is shown 3 pairs. We have 10 pairs of applicants; this means there are 30 possible combination (10*3) that could be shown to evaluators. If we collect 500 evaluators who each see 3 pairs, we can at most have 1500 possible observations, we then divide this by 2 for the gender known and gender not known treatment which gives us 750 obs. This means we have 25 observations for each possible pair combination (750/30 = 25). Total number of evaluator observations will be 1500.
Arms
Gender Known: 750 Gender Unknown: 750
Primary outcomes
We collect the following primary outcomes: - Choose to hire: this is defined as the applicant, out of a pair, that is chosen by the evaluator to be recommended for hiring by the evaluator
Secondary outcomes
To understand possible mechanisms we elicit the following secondary outcome variables: -Time spent: as a proxy for attention, we will measure how long evaluators spend on each pair. Greater time spent will be related to greater attention spent on making the right decision. -Gender: we will measure the likelihood that the chosen individual is female given whether gender is known and the gender composition of the pair -Quality: we will measure the relative quality of the applicant that is chosen, as measured by their qualifications, average evaluation scores from all evaluations given, and the AI-generated scores assigned to each applicant.

Cited by

No citation data available for this registration yet.

Keywords