Improving Hiring Decisions: Evidence on the Value of Information About Applicants Collected from their Professional References

How to cite

Goldhaber, Dan. 2025. "Improving Hiring Decisions: Evidence on the Value of Information About Applicants Collected from their Professional References." AEA RCT Registry. AEARCTR-0016575. https://doi.org/10.1257/rct.16575-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Dan Goldhaber*

* Primary investigator

Registry ID
AEARCTR-0016575
DOI
https://doi.org/10.1257/rct.16575-1.0
First registered
Last updated
Status
Completed
Trial dates
2015-06-22 – 2024-06-30
Countries
United States of America ()
JEL codes
M51; I21
Registry record
http://www.socialscienceregistry.org/trials/16575

Abstract

This work builds on prior research carried out under the U.S. Department of Education's Researcher-Practitioner Partnership grant program with Spokane Public Schools (SPS), which suggested that applicants’ professional references are a valuable source of information about applicant quality. Here, we aim to assess whether an enhancement to the process through which applicant data are collected from PRs can improve teacher selection. In particular, we will ask references to rate teacher applicants relative to their peers on a series of criteria demonstrated to be predictive of positive teacher outcomes and student achievement. The collection of letters of recommendation is already a standard procedure in the great majority of school districts and the collection of ratings data may represent relatively “low hanging fruit” as a means of improving teacher selection.

We analyze teacher selection in SPS during a four-year study period, focusing on teacher applicants to the district, and, specifically, on ratings data collected from references following the submission of a letter of recommendation. The reference ratings information is provided to SPS hiring officials for a random subset of applicants and available to use in the assessment of applicants during the hiring process.
This research design will allow us to address the following questions: (RQ1) What is the predictive validity of references’ assessments of applicants? (RQ2) To what extent do reference ratings data add to the informational content of the applicant data currently available to hiring officials? (RQ3) To what extent does the provision of reference ratings data influence hiring managers' assessments of applicants and hiring decisions?

The overall aim of the project is to provide an empirical foundation for understanding whether additional applicant data collected from applicants' references can be effectively used by a school district to make better hiring decisions. If so, this would provide a foundation for a more comprehensive study of the value of capturing insights about applicants from PRs, and could lead to the development of a tool that could be reliably and efficiently used by any school district.

Experimental design

We collect structured ratings of teacher job applicants from their professional references and provide this information to hiring managers for a randomized subset of applicants. Randomization is conducted at the applicant level (rather than at the job level) because it is common for applicants to apply for multiple positions during the course of the hiring season using a single application profile.

Design details

Randomization method
Randomization is done with a virtual coin flip using a computer.
Randomization unit
Individual.
Clusters
N/A
Observations
1,250 applicants to teaching positions.
Arms
625 applicants whose reference ratings information is provided to hiring managers and 625 applicants whose reference ratings information is withheld.
Primary outcomes
Measures of teacher applicants' progression through the hiring process: did an applicant advance past central screening; how did an applicant score on the rubric the district uses to determine which applicants to interview in person; was an applicant hired; if hired, what are the applicant's performance evaluation and retention outcomes.
Sponsors
Name: Institute for Education Sciences Url: https://ies.ed.gov

Papers from this trial

  1. Goldhaber, D., & Grout, C. (2024). Improving Hiring Decisions: Experimental Evidence on the Value of Reference Information about Teacher Applicants. Working Paper No. 306-0824. National Center for Analysis of Longitudinal Data in Education Research (CALDER). https://eric.ed.gov/?id=ED673326
  2. Goldhaber, D., Grout, C., Wolff, M., & Martinková, P. (2021). Evidence on the dimensionality and reliability of professional references’ ratings of teacher applicants. Economics of Education Review, 83, 102130. https://doi.org/10.1162/edfp_a_00421
  3. Goldhaber, D., Grout, C., Wolff, M., & Martinková, P. (2021). Evidence on the dimensionality and reliability of professional references’ ratings of teacher applicants. Economics of Education Review, 83, 102130. https://doi.org/10.1016/j.econedurev.2021.102130
  4. Goldhaber, D., & Grout, C. (2024). How Predictive of Teacher Retention Are Ratings of Applicants from Professional References? Working Paper No. 296-0324. National Center for Analysis of Longitudinal Data in Education Research (CALDER). https://eric.ed.gov/?id=ED673235

Cited by

No citation data available for this registration yet.

Keywords