Reference Letters as Incentives

How to cite

Garlick, Robert; Andrew, Alison; Field, Erica; Vyborny, Kate. 2026. "Reference Letters as Incentives." AEA RCT Registry. AEARCTR-0019327. https://doi.org/10.1257/rct.19327-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Robert Garlick* , Alison Andrew , Erica Field , Kate Vyborny

* Primary investigator

Registry ID
AEARCTR-0019327
DOI
https://doi.org/10.1257/rct.19327-1.0
First registered
Last updated
Status
On going
Trial dates
2026-01-01 – 2027-06-30
Countries
Pakistan ()
JEL codes
I25; I26; J24; O15
Registry record
http://www.socialscienceregistry.org/trials/19327

Abstract

Information frictions in the labor market can limit workers’ ability to signal their skills and may disproportionately affect women, who face statistical discrimination and limited access to professional networks. Prior research suggests that verifiable reference letters can improve labor market outcomes, particularly for women. Hence, we study whether the opportunity to earn a reference letter, which makes past work performance visible to employers, affects job selection, effort, performance, and skill investment.

We run a multi-round field experiment embedded in a one time online data-entry job. In Round 1, jobseekers responding to an online advertisement are randomized into one of two arms before deciding whether to sign up: (i) an offer of a verifiable reference letter or (ii) a control condition offering no reference letter. This round measures impacts on selection into the job.

In Round 2, participants who enrolled from the initial (Round 0) control group only are invited to interview for the job, and are re-randomized into two arms : (i) an offer of a verifiable reference letter, or (ii) control condition offering no reference letter, to measure effects on job effort, job performance, and skill investment during the job. If funding allows, we may include (iii) an offer of performance-based bonuses, to benchmark the value of the reference letter to participants.

Outcomes will be analyzed overall and across subgroups, including by gender, to examine heterogeneous treatment effects.

Intervention

The design includes two rounds:

Round 1:
Participants view an online advertisement for the job and, before deciding whether to sign up, are randomized into one of the two arms:
(i) Reference information, offering the opportunity to earn a verifiable reference letter
(ii) Control, offering no reference letter.

Round 2:
Only control-group participants from Round 1 who sign up are re-randomized into one of the two arms:
(i) Reference information, offering the opportunity to earn a verifiable reference letter
(ii) Control, offering no reference letter.
We may include a third treatment arm in round 2, offering performance-based bonuses in order to benchmark the magnitude of the reference effect. At the time of registering this trial, we are not yet sure if we will have sufficient funds to include this arm.

We will also collect information on baseline characteristics as part of the sign-up instrument, including gender, highest education qualification, current education status, current employment status, years of total experience, software-related skills, whether participants uploaded their CV, and whether they provided a reference letter for this job.

We will run analyses both with and without conditioning on covariates and will report whether the results differ. We will use a machine learning method such as random causal forests to assess if treatment effects differ by covariates.

Experimental design

The design includes two rounds:

Round 1:
Participants view an online advertisement for the job and, before deciding whether to sign up, are randomized into one of the two arms:
(i) Reference information, offering the opportunity to earn a verifiable reference letter
(ii) Control, offering no reference letter.

Round 2:
Only control-group participants from Round 1 who sign up are re-randomized into one of the two arms:
(i) Reference information, offering the opportunity to earn a verifiable reference letter
(ii) Control, offering no reference letter.
We may include a third treatment arm in round 2, offering performance-based bonuses in order to benchmark the magnitude of the reference effect. At the time of registering this trial, we are not yet sure if we will have sufficient funds to include this arm.

We will also collect information on baseline characteristics as part of the sign-up instrument, including gender, highest education qualification, current education status, current employment status, years of total experience, software-related skills, whether participants uploaded their CV, and whether they provided a reference letter for this job.

We will run analyses both with and without conditioning on covariates and will report whether the results differ. We will use a machine learning method such as random causal forests to assess if treatment effects differ by covariates.

Design details

Randomization method
Computer
Randomization unit
Treatment is assigned at the participant level for both rounds. For round 1, treatment is stratified by gender and location of the participant. For round 2, treatment is stratified by gender only. Analysis using panel data on participants will account for within-participant clustering.
Clusters
N/A
Observations
Round 1: 8984 participants Round 2: 2784 participants The final sample sizes may differ slightly depending on funding.
Arms
Round 1: - Offer of reference letter: 4492 - Control group: 4492 Round 2: - Offer of reference letter: 1392 - Control group: 1392
Primary outcomes
Round 1 outcomes: - Sign-up for the job (binary variable) Round 2 outcomes: - Job performance / accuracy - Job effort - Skill investment

Cited by

No citation data available for this registration yet.

Keywords