Understanding the Effect of Test-Optional Admissions Policies on Admissions

How to cite

Bleemer, Zachary; Friedman, John. 2022. "Understanding the Effect of Test-Optional Admissions Policies on Admissions." AEA RCT Registry. AEARCTR-0009777. https://doi.org/10.1257/rct.9777-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Zachary Bleemer* , John Friedman

* Primary investigator

Registry ID
AEARCTR-0009777
DOI
https://doi.org/10.1257/rct.9777-1.0
First registered
Last updated
Status
In development
Trial dates
2022-08-29 – 2024-07-01
Countries
United States of America (Northeast)
JEL codes
I23, I24
Registry record
http://www.socialscienceregistry.org/trials/9777

Abstract

In recent years, many colleges and universities are reconsidering their use of standardized test scores in their admissions processes. The debate has gained new urgency in recent years after COVID-19 forced most schools to adopt "test-optional" admissions policies. This study implements an experiment in which a selective northeast university's admissions officers re-evaluate real applications from past years with altered or omitted standardized test scores to determine how the presence or absence of test scores affects admissions decisions. The results from this experiment will answer the following questions: (1) How does a test-optional policy affect the way that admissions officers evaluate applications? (2) How do these effects vary across students, both by level of academic preparation and student background? In combination with descriptive results from other sources, these findings will characterize the winners and losers from test-optional policies and reveal the functional role of standardized tests in selective university admissions.

Intervention

The admissions office of a highly-selective private university in the northeast that currently implements a test-optional admissions policy will provide complete authentic undergraduate applications to its actual application readers. Each application will be provided to multiple readers, but with certain alterations: some will be provided with SAT scores above those actually received by the applicant, others with SAT scores lower than those actually received, and some will be provided applications with no SAT score at all, as if the applicant had chosen against submitting a score. Application readers will assign standard application scores to each application and record admission recommendations.

Experimental design

See the attached Experimental Protocol for our experimental design.

Design details

Randomization method
Randomization will be conducted in reseachers' office by a computer using Stata.
Randomization unit
Randomization will be conducted first at the geography level (choosing ten 'super-geos' by which the university's admissions readers evaluate applications), and then at the individual level with geography.
Clusters
20 application readers will evaluate applications.
Observations
About 500 applications will be evaluated.
Arms
Each application will be evaluated four times under the test-optional regime, in four treatment arms: large increase from actual standardized test score, small change in score, large decrease in score, and omission of the test score. Half of the selected applications will be evaluated four additional times under the test-mandatory regime, replacing the fourth arm with a chosen test score (see the experimental protocol).
Primary outcomes
The primary outcomes that we will measure for each application are: 1. An indicator for the applicant's being recommended to committee (a proxy for recommending admission). 2. An indicator for whether the applicant was recommended to committee or waitlist (as opposed to rejection). 3. The separate academic and non-academic reader scores assigned by the application readers. 4. An indicator for whether the applicant received a reader score (or each type separately) of four or greater.
Secondary outcomes
See the pre-analysis plan for an a more-detailed summary of additional outcomes.

Cited by

No citation data available for this registration yet.

Keywords