Using Predictive Analytics to Track Students: Evidence from a Seven-College Experiment

How to cite

BERGMAN, PETER; Kopko, Elizabeth; Rodriguez, Julio. 2021. "Using Predictive Analytics to Track Students: Evidence from a Seven-College Experiment." AEA RCT Registry. AEARCTR-0007827. https://doi.org/10.1257/rct.7827-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

PETER BERGMAN* , Elizabeth Kopko , Julio Rodriguez

* Primary investigator

Registry ID
AEARCTR-0007827
DOI
https://doi.org/10.1257/rct.7827-1.0
First registered
Last updated
Status
Completed
Trial dates
2016-05-01 – 2021-06-01
Countries
United States of America (New York)
JEL codes
I20, I24
Registry record
http://www.socialscienceregistry.org/trials/7827

Abstract

Tracking is widespread in U.S. education. In post-secondary education alone, at least 71% of colleges use a test to track students. However, there are concerns that the most frequently used college placement exams lack validity and reliability, and unnecessarily place students from under-represented groups into remedial courses. While recent research has shown that tracking can have positive effects on student learning, inaccurate placement has consequences: students face misaligned curricula and must pay tuition for remedial courses that do not bear credits toward graduation. We develop an alternative system to place students that uses predictive analytics to combine multiple measures into a placement instrument. Compared to colleges’ existing placement tests, the algorithm is more predictive of future performance. We then conduct an experiment across seven colleges to evaluate the algorithm’s effects on students. Placement rates into college-level courses increased substantially without reducing pass rates. Adjusting for multiple testing, algorithmic placement generally, though not always, narrowed gaps in college placement rates and remedial course taking across demographic groups. A detailed cost analysis shows that the algorithmic placement system is socially efficient: it saves costs for students while increasing college credits earned, which more than offsets increased costs for colleges. Costs could be reduced with improved data digitization, as opposed to entering data by hand.

Intervention

See paper: http://www.columbia.edu/~psb2101/bergmananalytics.pdf

Experimental design

See paper: http://www.columbia.edu/~psb2101/bergmananalytics.pdf

Design details

Randomization method
Computer
Randomization unit
Individual
Clusters
n/a
Observations
12,544
Arms
6,141 control 6,403 treatment
Primary outcomes
See paper.

Cited by

No citation data available for this registration yet.

Keywords