Disentangling Sources of Bias: Evidence from Advanced Placement Course Recommendations

How to cite

Francis, Dania; Oliveira, Angela de; Dimmitt, Carey. 2021. "Disentangling Sources of Bias: Evidence from Advanced Placement Course Recommendations." AEA RCT Registry. AEARCTR-0007927. https://doi.org/10.1257/rct.7927-5.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Dania Francis* , Angela de Oliveira , Carey Dimmitt

* Primary investigator

Registry ID
AEARCTR-0007927
DOI
https://doi.org/10.1257/rct.7927-5.0
First registered
Last updated
Status
In development
Trial dates
2023-09-01 – 2024-09-01
Countries
Private
JEL codes
D90, I24, J15, J16
Registry record
http://www.socialscienceregistry.org/trials/7927

Abstract

In this project, we seek to understand minority and female underrepresentation in advanced STEM courses in high school by investigating whether school counselors exhibit racial or gender bias during the course assignment process. We extend the analysis by attempting to disentangle whether any observed bias can be attributed to taste-based discrimination, statistical discrimination, or implicit bias. Using an adapted audit study, we asked a nationally-recruited sample of school counselors to evaluate student transcripts that were identical except for the names on the transcripts, which were varied randomly to suggestively represent a chosen race and gender combination. In order to identify the sources of bias we included three additional experimental conditions. Understanding the underlying sources of racial and gender bias can help stakeholders and policymakers design better solutions to address the bias.

Intervention

The plan for this study is to examine whether school counselors are more or less likely to recommend student (as represented by their academic transcripts) for advanced placement (AP) calculus courses based on the race and gender suggested by the student names, and how prepared the school counselor believes the student would be for the AP calculus course. If there is race or gender bias in the likelihood of recommendation or the preparedness rating, is it possible to identify the source of the bias (statistical discrimination, implicit bias, or taste-based discrimination) by implementing experimental interventions and survey based measurements of those sources?

Experimental design

To answer the proposed research questions, we conduct a randomized experiment that is a modification of a correspondence audit study. We ask study participants to evaluate a set of 6 profiles which are given at random either a Black, White, Hispanic, or Asian sounding name, or no name (blind review). There are also typical male or female sounding names within each racial/ethnic category. For the Black and White-sounding names, we also choose names indicative of high, medium, or low socioeconomic status as demonstrated by sociological research on names.

Design details

Randomization method
The randomization will be conducted by Qualtrics survey software .
Randomization unit
Individual. There are two levels of randomization. Assignment to blinded transcripts without PSAT scores (control), blinded transcripts with PSAT scores (Treatment A), race/gender transcripts without PSAT scores (Treatment B) or race/gender transcripts with PSAT scores (Treatment C) will be randomized. Once assigned to control, treatment A, treatment B, or treatment C, the type of transcript presented for each of the 4 treatment transcripts (strong or borderline) will be randomized and a for those in the race/gender treatments, a randomized race/gender signifying name will be paired with the randomized transcript.
Clusters
1300 School Counselors
Observations
5232 transcript ratings over 1308 School Counselors
Arms
Blinded transcripts without PSAT scores - 65 counselors (260 transcripts) Blinded transcripts with PSAT scores - 65 counselors (260 transcripts) Race/gender transcripts without PSAT scores - 589 counselors (2356 transcripts) Race/gender transcripts with PSAT scores - 589 counselors (2356 transcripts)
Primary outcomes
1) recommendation for AP Calculus (yes or no) 2) preparedness for AP Calculus (scale of 1-10)

Cited by

No citation data available for this registration yet.

Keywords