The labor market implications of grade inflation - manager experiment

How to cite

Carpenter, Jeffrey; Abel, Martin; Pu, Zhizhong. 2025. "The labor market implications of grade inflation - manager experiment." AEA RCT Registry. AEARCTR-0016402. https://doi.org/10.1257/rct.16402-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jeffrey Carpenter* , Martin Abel , Zhizhong Pu

* Primary investigator

Registry ID
AEARCTR-0016402
DOI
https://doi.org/10.1257/rct.16402-1.0
First registered
Last updated
Status
In development
Trial dates
2025-07-18 – 2025-07-31
Countries
Private
JEL codes
J24, J31, C90
Registry record
http://www.socialscienceregistry.org/trials/16402

Abstract

We design an experiment to study the implications of grade inflation for hiring and wage setting. Our research questions concern how compressed and inflated grades affect the perceived informativeness of letter grades in a coarse signaling setting where underlying math ability is unobserved. Specifically, we investigate how these different grading schemes affect the signals participants infer from letter grades on a math test and how this information, along with their priors, factors into their assessment of the ability of job candidates. We also study how coarsening grade signals affects gender wage discrimination.

Experimental design

The experiment will be conducted online (using Connect or Prolific) and will consist of two components, each lasting about 15 minutes, run concurrently (i.e., within a week of each other). In both components, participants are told to imagine themselves in the role of a manager looking to hire workers for their firm where underlying ability on the job is measured by the math portion of the SAT exam. Their task is to evaluate the profiles of and set wages for a group of candidates who participated in a separate study.

The manager experiment consists of participants first reporting their priors about how scores on the math SAT are distributed in the candidate population. Managers learn that the candidates have taken a math test and that, consistent with college transcripts, they will see the letter grades candidates received on the test, but not the raw scores. After seeing three sample questions from the test, managers are asked for their expectations about how hard the test was (i.e., how well candidates have done).

Managers then learn that they will be randomly assigned to one of three grading treatments: a control in which a college professor scores the tests and assigns letter grades from C- to A+ (i.e., 9 grading bins), compress where a professor scores the tests but only assigns grades from B- to B+ (3 bins) and inflate where a professor only assigns grades from A- to A+ (also 3 bins).

As a signal extraction exercise, managers are first asked to predict the grading scheme used by the professor in the treatment to which they are assigned. Specifically, they are asked to report the lowest numerical score (from 0 to 100 percent) needed to receive each letter grade. Second, they are randomly shown 9 of the 18 possible candidate profiles (which include the letter grades received on the test) and asked to assign a numerical score between 0 and 100 to each. Managers are incentivized to report scores that match the true test performance of the candidates. These scores are considered ability \emph{signals} in the analysis described below.

In the last part of the experiment, managers see expanded profiles for the same 9 candidates and are incentivized to assign wages that match their ability (i.e., math SAT scores). Here, the profiles include the math SAT priors the candidates reported first and the letter grades each candidate received on the test to provide managers with the information a Bayesian would require to set wages optimally.

Managers then answer a few demographic questions before being redirected back to the platform for payment. They receive a fixed payment for completing the experiment and can earn a bonus for reporting correct priors about the average math SAT scores of the candidates, their estimates of the grading schemes and performance on the math test, assigning scores that match the actual performance of candidates on the test and assigning wages that match individual candidate math SAT scores and their variance estimates.

Managers in both components of the experiment know that all the candidates are between 21 and 25 years old, have recently graduated from a four-year college/university (or are enrolled in one) and have taken the SAT exam. The only difference between the first component of the experiment and the second is that candidate gender is also revealed in the first component, and it is not revealed in the second component.

Design details

Randomization method
In office by computer
Randomization unit
Experimental participant
Clusters
between 800 and 2100 participants
Observations
Each participant evaluates nine candidate profiles each so between 7200 and 18900.
Arms
The sample will be divided equally among the three treatment conditions.
Primary outcomes
Incentivized scores and wages

Cited by

No citation data available for this registration yet.

Keywords