Algorithmic RCT in Child Protection Services

How to cite

Ghosh, Arkadev; Baron, Jason; Lombardo, Richard. 2023. "Algorithmic RCT in Child Protection Services." AEA RCT Registry. AEARCTR-0012623. https://doi.org/10.1257/rct.12623-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Arkadev Ghosh* , Jason Baron , Richard Lombardo

* Primary investigator

Registry ID
AEARCTR-0012623
DOI
https://doi.org/10.1257/rct.12623-1.1
First registered
Last updated
Status
Completed
Trial dates
2023-12-13 – 2025-06-01
Countries
United States of America ()
JEL codes
I38
Registry record
http://www.socialscienceregistry.org/trials/12623

Abstract

Every year, around 4 million referrals are made to Child Protective Services in the United States, involving more than 7 million children. Approximately half of these reports are screened in for investigation. Screened-in referrals are assessed by a caseworker, with support from their supervisor. There are few tools that are available to supervisors to support their practice and encourage rich conversations with caseworkers. A county in the U.S. is planning to launch the CSDA Supervision Tool, a data lead tool developed by professors at UNC-Chapel Hill and Auckland University of Technology and Centre for Social Data Analytics (CSDA). This study will conduct an impact evaluation to assess whether these tools are (i) improving outcomes for families and (ii) not having any adverse outcomes. A key feature of the study will be its focus on treatment heterogeneity by workers' baseline beliefs about the risk distribution of different demographic groups.

Intervention

The CPS agency is launching the CSDA Supervision Tool, a data lead tool developed by professors at UNC-Chapel Hill and Auckland University of Technology and Centre for Social Data Analytics (CSDA). The goal of the tool is to optimally target agency resources to child maltreatment investigations based on their risk profile.

Experimental design

The RCT is designed to estimate the causal effects of the supervision tool on the agency's decisions and children's subsequent outcomes. Randomization will take place at the household level. The first time the agency receives a report for a given family, the supervisor will be randomly shown the family's risk score or not. The supervisor will then communicate this score to the investigator/caseworker, who will consider the score when deciding the appropriate course of action during the investigation/assessment. Families will keep their initial randomization status throughout the trial period.

Design details

Randomization method
Randomization is done in a computer at the agency's office.
Randomization unit
Household
Clusters
This will depend on how many cases arrive in the following year. Our best guess is that, in this county, we will see about 1000-3000 households during our trial period.
Observations
We estimate that any given household will have an average of two children. Any given report may contain information on other children in the household, besides the alleged victim. Therefore, we expect roughly 2000-6000 observations.
Arms
Roughly 500-1500 clusters in each group (treatment versus control).
Primary outcomes
Measure of safety using CPS data: re-referrals to the agency, subsequent substantiated investigations, subsequent foster care placements, racial disparities in safety and agency responses. For each of these outcomes, we will get the data directly from the CPS agency. We will also conduct a survey at multiple points of the RCT, in which we will assess workers' beliefs regarding the risk profile of different demographic groups (by race and by gender)---and how these beliefs may be changing as a result of the intervention.
Secondary outcomes
(1) Variance of service provision across investigators. We also *hope* to get external data measuring safety such as hospitalization records. (2) Heterogeneous treatment effects by investigators' characteristics, including observable traits -- particularly, gender, family characteristics, and baseline trust in the algorithm. (3) "First stage" outcomes such as provision of prevention services, time spent on investigations, face-to-face interaction with the family's home, and effort during investigations.

Cited by

No citation data available for this registration yet.

Keywords