Wiseguys or Wise Guys? How do high school equivalency credentials affect third-party appraisals of job applicants' resumes?

How to cite

Heller, Blake; Singla, Henam. 2025. "Wiseguys or Wise Guys? How do high school equivalency credentials affect third-party appraisals of job applicants' resumes?." AEA RCT Registry. AEARCTR-0015269. https://doi.org/10.1257/rct.15269-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Blake Heller* , Henam Singla

* Primary investigator

Registry ID
AEARCTR-0015269
DOI
https://doi.org/10.1257/rct.15269-1.1
First registered
Last updated
Status
In development
Trial dates
2025-03-12 – 2025-04-15
Countries
Private
JEL codes
I20, I26, J24, J71
Registry record
http://www.socialscienceregistry.org/trials/15269

Abstract

This study will measure the influence of reported educational attainment on third-party evaluations of job applicants, focusing the penalties or rewards associated with applicants' reporting having earned a high school equivalency credential. We will recruit roughly 1,000 hiring managers from the five states (CA, FL, GA, NY, and TX) with the largest number of GED testers in 2022 and ask them to evaluate resumes of applicants with randomly assigned characteristics, including implied gender, implied race/ethnicity, work experience, volunteer experience, and educational attainment. We will assess the impact of observing different educational credentials on these hiring managers' appraisals of the resumes, focusing on the relationship between observing a high school equivalency credential and applicant ratings relative to 1) no educational credential reported, 2) a traditional high school diploma (from high- and low-achieving schools), and 3) some college. Additionally, a subset of applicants who report having a high school equivalency credential will also report earning a "GED College Readiness" designation in math or language arts. Finally, we during the survey, will present two different entry-level job descriptions to evaluators, asking them to assess five pairs of resumes for each type of job: a customer service position (with relatively high social skill task content) and a data entry position (with relatively low social skill task content). We will randomize the order in which job descriptions are presented so that some evaluators assess applicants for a customer service position first versus the data entry job (and vice versa), allowing us to test whether the impact of educational credentials is related to the type of job for which applicants are applying.

Intervention

We will recruit roughly 1,000 evaluators with managerial experience from five states (CA, FL, GA, NY, and TX) and ask them to evaluate resumes of applicants with randomly assigned characteristics, including implied gender, implied race/ethnicity, work experience, volunteer experience, and educational attainment. We will also randomly assign the entry-level job description that is presented to evaluators between a customer service position with relatively high social skill task content versus an administrative job with relatively low social skill task content and test whether the impact of educational credentials is related to job characteristics.

Before collecting data from all 1,000 evaluators, we will run a pilot of 50 evaluators to test survey functionality. Once we confirm that they survey works as intended, we will scale up data collection to the remaining 950 evaluators.

We will assess the impact of observing different resume characteristics on these hiring managers' appraisals of the resumes.

Our primary analysis will focus on assessing the relationship between observing a high school equivalency credential and evaluator appraisals relative to applications with 1) no educational credential reported, 2) a traditional high school diploma, and 3) some college.

Our exploratory analyses will assess whether there are differential impacts of listing a high school equivalency credential on evaluator rating for the subset of applicants who report having a high school equivalency credential and also report earning a "GED College Readiness" designation in math or language arts as well as assessing whether the impact of educational credentials varies by job type (customer service vs. data entry). We will also examine whether different resume characteristics moderate the effect of educational credentials on evaluations (e.g., more or less experience; racial or gender differences in impacts (as well as their interaction); type of experience [volunteer vs. paid work; sector of paid work], etc.). We will present our primary analyses pooled across cities, but also publish results separately by state. Additionally, we will assess whether earning a HS diploma from a high-performing vs. low-performing high school moderates any gap in evaluation between HSE equivalency credential holders and traditional HS grads, with and without limiting our sample to evaluators who accurately identify at least 2 of 3 high performing high schools (or 2 of 3 low-performing high schools) in their metro area. Finally, we will assess the mechanisms that explain why hiring managers reward or penalize applicants with HSE by examining whether participants ratings of applicants or preferences for applicants with HSE credentials varies with the factors they report considering as they evaluated candidates as well as the factors they state are "most important" for hiring managers to consider (e.g., social skills, problem solving skills, language skills, alignment of work experience with job, criminal history, etc.).

Experimental design

The experimental design involves recruiting 1,000 hiring managers from five states (CA, FL, GA, NY, TX) to evaluate resumes with randomly assigned characteristics. Each resume will have variations in implied gender, implied race/ethnicity, work experience, volunteer experience, and educational attainment (GED, high school diploma, no education, or some college). Additionally, job descriptions will be randomly assigned to either a customer service role (with high social skill content) or an administrative role (with low social skill content). Hiring managers will rate each resume on various attributes and choose a preferred candidate from pairs of resumes, allowing us to assess the impact of educational credentials on their evaluations. A subset of applicants who report having a high school equivalency credential will also report earning a "GED College Readiness" designation in math or language arts.

Design details

Randomization method
The randomization of the order in which job types are presented to evaluators is implemented using the randomizer function in Qualtrics. The randomization of which resume pairs are presented to evaluators (and their order) is implemented using the "Loop and Merge" function in Qualtrics where 1800 candidate pairs are randomly sorted and divided evenly among modules (180 per module), and evaluators are presented with 1 pair out of these 180 options for each of 10 comparisons. The randomization of characteristics to resumes is done in office by a computer to assign educational credentials and job descriptions to the resumes (e.g., by assigning each observation a uniform random number and assigning a particular characteristic in proportion to the number of realized values of that characteristic). This ensures that each characteristic is randomly allocated, minimizing bias and allowing for the unbiased assessment of the impact of educational credentials on evaluator appraisals.
Randomization unit
The unit of randomization is the individual resume. Each resume will have randomly assigned characteristics, such as implied gender, implied race/ethnicity, work experience, volunteer experience, and educational attainment (GED, high school diploma, no education, or some college). Evaluators will be randomly assigned pairs of resumes to assess, allowing for a within-subject, individual-level randomization.
Clusters
This is a within-subject experiment, so the randomization is clustered within 1000 respondents from 5 cities. Our primary results will include an evaluator fixed effect to account for idiosyncratic differences in evaluators' average scores, with standard errors clustered by evaluator. Evaluators in different states will rate different pools of applications (e.g., Texas evaluators will be presented with a hypothetical candidate applying for a job in Houston, Texas while California evaluators will be presented with a hypothetical candidate applying for jobs in Los Angeles, CA). Each hypothetical job search scenario is constructed around a candidate applying for a customer service or data entry job in that state's largest metro area (CA: Los Angeles; FL: Miami; GA: Atlanta; NY: New York City; TX: Houston). All analyses will also include a job type fixed effect.
Observations
The planned number of observations is 20,000 resumes (1,000 evaluators × 20 resumes each).
Arms
1,000 evaluators evaluating 20 resumes each (20,000 resume ratings/selections) for primary outcomes 1 and 2.
Primary outcomes
1) Overall evaluator scores of resumes. Evaluators will be asked to rate each candidate overall on a scale of 1-10. These scores will be standardized to have a mean of zero and a standard deviation of one. We will assess the impact of resume characteristics on primary outcome 2 using ordinary least squares regression with evaluator fixed effects, clustering our standard errors at the evaluator level. 2) The probability that an Evaluator selects a resume with a given realized value of a characteristic, conditioning on the realized values of all other resume characteristics. We will assess the impact of resume characteristics on primary outcome 2 using logistic regression with evaluator fixed effects, clustering our standard errors at the evaluator level.
Secondary outcomes
Evaluator sub scores of resumes: Evaluators will rate each candidate on specific dimensions such as critical thinking & problem-solving and grit & persistence. These ratings will be standardized to have a mean of zero and a standard deviation of one, ensuring uniformity in how the sub scores are compared across all resumes evaluated. We will also examine the impact of reporting a HSE credential on each component of the primary rating index and use these subscores to test whether reporting an HSE credential influences evaluators’ appraisal of candidates’ critical thinking and problem solving skills (subindex 1: academic skills) differently than their appraisal of candidates’ grit and persistence (subindex 2: non-academic skills) as predicted by Araujo, Gottlieb, and Moreira (2007).

Cited by

No citation data available for this registration yet.

Keywords