Algorithmic Accuracy by Group and Recidivism Prediction

How to cite

Lee, Yong Suk. 2023. "Algorithmic Accuracy by Group and Recidivism Prediction." AEA RCT Registry. AEARCTR-0010807. https://doi.org/10.1257/rct.10807-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Yong Suk Lee*

* Primary investigator

Registry ID
AEARCTR-0010807
DOI
https://doi.org/10.1257/rct.10807-1.0
First registered
Last updated
Status
In development
Trial dates
2023-01-31 – 2023-02-28
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/10807

Abstract

This study examines how AI performance metrics affects police prediction of re-arrests using an online randomized survey. Studies have found that base commercial risk assessment softwares are no more accurate or fairer than predictions made by people with little or no criminal justice expertise. However, algorithms do outperform human predictions of recidivism when enriched set of risk factors are included in the model. This study examines how police predication compares with algorithms and whether predictions differ when race/ethnicity is provided, and the accuracy of the algorithm by race/ethnicity.

Intervention

This study examines how AI performance metrics affects police prediction of re-arrests using an online randomized survey. A representative sample of the US population born between 1980 and 1983 with arrest histories are randomly presented to respondents. Respondents are asked to predict the likelihood that this person will be arrested again in the next three years. Respondents are then shown an AI algorithm's recidivism prediction score for each profile and allowed to adjust their prediction. Different treatment groups will be presented with a combination of information related to race and ethnicity, as well as the AI algorithm's prediction accuracy.

Experimental design

Each respondent will see 20 profiles and make predictions on each.
Base group: Will have no race/ethnicity information
Treatment Group 1 : Will be shown race/ethnicity information
Treatment Group 2 : Will be shown the AI prediction accuracy by race/ethnicity
Treatment Group 3: Will be shown race/ethnicity information plus the AI prediction accuracy by race/ethnicity

Design details

Randomization method
online randomization using survey company
Randomization unit
individuals
Clusters
no clusters
Observations
150
Arms
50 no race, 50 race, 50 mixed
Primary outcomes
prediction probability of re-arrests

Cited by

No citation data available for this registration yet.

Keywords