Labor Market Returns to Upskilling - A Combination of Audit Study and Resume Review

How to cite

Zhu, Zhengren; Sinha, Sourav. 2020. "Labor Market Returns to Upskilling - A Combination of Audit Study and Resume Review." AEA RCT Registry. AEARCTR-0005718. https://doi.org/10.1257/rct.5718-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Zhengren Zhu* , Sourav Sinha

* Primary investigator

Registry ID
AEARCTR-0005718
DOI
https://doi.org/10.1257/rct.5718-1.0
First registered
Last updated
Status
In development
Trial dates
2021-06-01 – 2021-09-01
Countries
Private
JEL codes
J01, J24, J62, I26
Registry record
http://www.socialscienceregistry.org/trials/5718

Abstract

The US labor market has experienced significant shifts over the last few decades due to skill-biased technological changes (Autor et al., 2010). This polarization of employment opportunity has exacerbated income inequality across the country, marginalizing a large fraction of the workforce, who do not have the background and resources to pursue high-skilled careers through higher education. One potential solution is to retrain displaced workers and disadvantaged population into technological industries with considerable growth potential. However, there is a lack of evidence on what type of retraining programs are effective in helping disadvantaged groups gain foothold in high-skilled industries. This research project aims to estimate the effectiveness of entry-level online tech certificates in helping individuals move from other sectors into the tech industry. In addition, this project asks whether online tech certificates are equally valuable, more valuable or less valuable for workers without relevant tech-experience and education. Finally, this project also asks whether age interacts with the returns to tech certificates.

Credibly estimating the labor market returns to online certificates poses significant challenges. Many retraining programs (eg. Year Up) have strict admission guidelines for participants, and often work closely with corporate partners to place participants at the end of the program. Therefore, simply comparing income of individuals with and without certificates would lead to biased estimates, because workers who choose to attain certificates are inherently different from those who do not. To address these challenges, this project uses a combination of audit study and Incentivized Resume Rating (IRR) methods. The core of the research design is an audit study that elicits employers’ true preferences for IT certificates. In this step, we first carefully design artificial job applications and randomly assign part of the application profiles to have selected online tech certificates. Then, the resumes are sent out to job openings posted on a large online job search platform in the US, and call-back responses are analyzed. A subset of these artificial applications are further subjected to incentivized resume rating, where third-party resume evaluators decide whether they would call-back these applicants and what salary they would be offered. By analyzing the results on call-back rates from the IRR and audit studies, we first establish a correspondence between the weights that resume evaluators and real employers assign to various observable worker characteristics. Then, this correspondence is applied to resume reviewers' behavior in the IRR salary study to find the expected labor market returns to skill certificates in a hypothetical audit study on salary.

Results from this project would inform policy-makers on the viability of using certifications and related training programs in assisting displaced workers move into the tech industry. Given the increasing rate of technological advancement and changing nature of jobs, these results have potential to guide policies that can reduce income inequality and at the same time create a pool of high-skilled workforce. This project would generate rich datasets on worker profiles typical of young and entry-level job applicants in the tech industry, call-back rates from the audit and IRR studies, and expected offers from the IRR salary study. Summary statistics from this dataset will be of independent interest to organization that are invested in designing and administering online skill certification programs. Methodologically, this project provides the first direct examination of whether third-party resume reviews are a viable alternative to audit studies, and proposes a new method to utilize the relative advantages of both audit study design and resume review studies.

Experimental design

This project uses a combination of audit study and Incentivized Resume Rating (IRR) methods. The core of the research design is an audit study that elicits employers’ true preferences for IT certificates. In this step, we first carefully design artificial job applications and randomly assign part of the application profiles to have selected online tech certificates. In addition, we randomly assign the applicant to either have or not have previous tech-related experience, and also randomly assign the applicant to either be young (~22 yo) or older (~35 yo). Then, the resumes are sent out to job openings posted on a large online job search platform in the US, and call-back responses are analyzed. A subset of these artificial applications are further subjected to incentivized resume rating, where third-party resume evaluators decide whether they would call-back these applicants and what salary they would be offered. By analyzing the results on call-back rates from the IRR and audit studies, we first establish a correspondence between the weights that resume evaluators and real employers assign to various observable worker characteristics. Then, this correspondence is applied to resume reviewers' behavior in the IRR salary study to find the expected labor market returns to skill certificates in a hypothetical audit study on salary.

Design details

Randomization method
Randomization is implemented with random resume generator (Lahey and Beasley, 2007)
Randomization unit
Individual
Clusters
12000 individual applications
Observations
12000 individual applications
Arms
6 types of resumes: no tech experience + no certificate + young, no tech experience + with certificate + young, with tech experience + no certificate + young, with tech experience + with certificate + young, no tech experience + no certificate + older, no tech experience + with certificate + older. 2000 individual applications in each arm. Total of 12000 individual applications.
Primary outcomes
Call back response for audit study. Reviewer decision on call back, job offer, and expected wage.

Cited by

No citation data available for this registration yet.

Keywords