AI-Integrated Tools for Improved Civil Prosecution Outcomes in Trafficking Cases: a Cluster-Randomized Controlled Trial

How to cite

Babiarz, Kimberly; Miller, Grant; Seiler, Ben. 2026. "AI-Integrated Tools for Improved Civil Prosecution Outcomes in Trafficking Cases: a Cluster-Randomized Controlled Trial." AEA RCT Registry. AEARCTR-0018894. https://doi.org/10.1257/rct.18894-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Kimberly Babiarz* , Grant Miller , Ben Seiler

* Primary investigator

Registry ID
AEARCTR-0018894
DOI
https://doi.org/10.1257/rct.18894-1.0
First registered
Last updated
Status
In development
Trial dates
2024-10-01 – 2027-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/18894

Abstract

This document outlines the analysis plan for a cluster-randomized controlled trial of an AI tool to assist prosecutors pursuing labor trafficking cases in Brazil, a tool complementing and working with an AI-native decision support tool Bússola I.A.. The experiment will test whether federal labor prosecutors using the tool to identify the most promising cases, compile supporting documentation,and prepare prosecutions faster yields improved prosecution outcomes. This pre-analysis plan provides intervention details, methodology, and a plan for analyzing the results of the experiment.

Intervention

The proposed intervention seeks to answer the following research question: Can LLM-driven decision
support technologies for anti-trafficking prosecutors improve prosecution outcomes in labor trafficking
cases? We evaluate the impact prosecution-related features of Bússola I.A.have on downstream civil
and criminal prosecution outcomes. Specifically, we will measure impacts on one prosecution-related
primary outcome: the total size of civil judgments (across all types of civil penalties and fines)
imposed on trafficking cases, and one secondary outcome: the probability that a trafficking case
within an FLPO jurisdiction is resolved through civil or criminal prosecution.

Bússola I.A. is a modular tool with features that fall into two categories ultimately related to the
successful prosecution of trafficking cases: core functionalities designed to improve and expedite
trafficking case investigations, and features to assist in post-investigation prosecution process.

Features for improved investigation
First, Bússola I.A. performs near real time risk scoring to identify and prioritize the tips and cases
most likely to lead to trafficking discovery and prosecution. To do so, the model uses a nimble, locally
implemented LLM to process the information contained in the incoming tip, which is submitted in free
text narrative format. The tool organizes extracts key features from tip received that are relevant for
decision making, organizing these features into a structured dataset (for example: whether violence is
reported, whether children are mentioned, whether an address is provided, economic sector). These
features are then used in a machine learning model to estimate the likelihood of trafficking discovery
and the priority level of the tip. This method aims to reduce reliance on subjective intuitions or
availability-bias in the early stages of case prioritization, leading to more effective case prioritization
and improved task force targeting.

Second, Bússola I.A. uses Large Language Models (Gemini and GPT4.1) to produce concise text-
summary of the key features of the tip, along with the primary components driving the tip’s risk score
and important features (ie possible child labor, violence or weapons, sexual exploitation, relevant
economic sector etc.). The goal of the LLM-generated summaries is to provide a quick, high-level
summary to prosecutors, and to describe the main components driving the risk score corresponding
to the tip. These features are critical for quickly conveying key pieces of information in an easy-to-
understand and interpret narrative framework, as well as explaining the risk score prioritization.
The Bússola I.A.interface translates these key features into tags that can be used to filter and sort
open cases according to priority or common features. The LLM augmentation is designed to expedite
tip processing for faster desk investigations and more timely task force responses.

Third, Bússola I.A. visualizes the spatial distribution of all active tips in an interactive map showing
the locations and characteristics of each active case. This feature allows prosecutors examining a
given case to view all cases originating from nearby locations, in similar sectors, or the most recent
cases. The purpose of the feature is to facilitate the identification of clusters of tips, identifying
opportunities for case bundling, leading to larger task forces which investigate more cases.

Features to facilitate post-investigation prosecution
First, Bússola I.A.embeds an AI tool that interprets and describes evidence collected during investiga-
tions - describing the contents of photographs and providing relevant legal codes, and summarizing
the contents of depositions and worker interviews. The aim of this feature is to simplify and speed
the work of prosecutors in drafting detailed lawsuits and complaints.

Second, Bússola I.A.includes document preparation tool that populates standard format documents
and petitions related to prosecution. This function brings together the most critical features of the case,
narrative summaries of the circumstances, evidence summary narratives, relevant legal violations
and corresponding legal codes, and attaching photographic evidence where available. The tool can
be used to produce draft documents needed to petition traffickers for settlement agreements, file
lawsuits, and appeal judicial decisions. The purpose of this tool is not to automate these processes, but
to populate first draft documents that can speed the filing process, and to ensure completeness.

The Bússola I.A. tool places a strong emphasis on interpretability and transparency allowing for
real-time interrogations of model outputs to build trust with human decision makers. Moreover, the
Bússola I.A. tool is designed to complement and integrate into the existing case management system
MPT Digital as a dashboard extension, minimizing changes in workflow required to access Bússola
I.A. analytics. In other words, Bússola I.A. is designed to be a new feature of the existing MPT Digital
platform rather than a completely new replacement.

Experimental design

Our primary analysis is on the task force level, and task forces are clustered within jurisdiction. The 23 included clusters (i.e., jurisdictions) are placed in randomization-blocks formed by whether the jurisdiction is above or below median historical case load (using the case load records for each office over the last ten years). Given the 23 clusters, the median split puts 12 in the below and 11 in the above groups. The cluster-level randomization then occurs within blocks, with treatment:control ratios of 6:6 for the below the median block and 5:6 for the above-the-median block.

Design details

Randomization method
Cluster randomization done before implementation by a computer
Randomization unit
Prosecutorial jurisdictions (roughly corresponding to Brazilian states) are randomized into either a treatment or control arm.
Clusters
23 Clusters
Observations
At least 700 trafficking cases
Arms
11 treatment jurisdictions 12 control jurisdictions
Primary outcomes
Our primary outcome is defined as the total monetary value of civil penalties imposed for each case for which trafficking is discovered. Discovery of trafficking may result in a settlement agreement negotiated directly between prosecutors and defendants or lawsuits (including preparatory lawsuits) filed following desk investigations or task force investigations. The result of lawsuits and settlement agreements may be 1) mandatory actions to address conditions, 2) actions the defendant firm or individual is barred from undertaken, and 3) fines and penalties assessed directly (including moral damages, worker restitution, punitive damages, and seized assets). We sum broadly across the monetary value of these actions, summing across all types of fines and penalties and valuing non-monetary obligatory actions based on the monetary value of the stipulated fine for failing to meet the obligation. Study units are trafficking cases investigated by authorities, where trafficking is determined. Because our tool is initiated when a trafficking-related case is formed from an incoming tip, the flow of cases available for investigation is assumed to be exogenous to the experiment. Historical data suggests the average number of cases prosecuted in each jurisdiction in each year is about 45-46 cases (out of a total of 136 cases per year).
Secondary outcomes
Because our tool may increase the probability that a case will be processed successfully to prosecutorial action, but not all prosecution-related outcomes can be assigned a monetary value (criminal prosecution or Dirty List inclusion, for example), we propose studying the case prosecution rate as a secondary outcome. Prosecution rate is defined as the total number of cases litigated in civil court using any civil prosecution strategy (lawsuits for damages, settlement agreements, asset seizure, Dirty List inclusion, etc.) or cases prosecuted in criminal courts as a share of the total number of cases opened in a given jurisdiction in a given year. Trafficking cases that are prosecuted in both civil and criminal justice systems will be counted as a single study unit. We note that criminal prosecution is rare in Brazil, with just a few cases pursued each year, we do not have a sufficiently large sample to study criminal prosecution rates formally.

Cited by

No citation data available for this registration yet.

Keywords