Effect of Court Auditors on Judicial Decisions

How to cite

Coviello, Decio; Benson, Lenni; Persico, Nicola; Todd, Petra. 2023. "Effect of Court Auditors on Judicial Decisions." AEA RCT Registry. AEARCTR-0011751. https://doi.org/10.1257/rct.11751-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Decio Coviello* , Lenni Benson , Nicola Persico , Petra Todd

* Primary investigator

Registry ID
AEARCTR-0011751
DOI
https://doi.org/10.1257/rct.11751-1.0
First registered
Last updated
Status
In development
Trial dates
2023-07-10 – 2024-07-10
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/11751

Abstract

Every day in immigration courts across the US, judges hold hearings to determine whether the foreign nationals in their dockets (technically called respondents) should be removed from the country or, instead, allowed to stay. In these hearings, judges have latitude in choosing procedural steps that may, eventually, result in the foreign nationals being removed. The goal of this research is to assess whether judges respond to being observed by law students and other trained observers and, in particular, whether being observed makes a judge more motivated to provide flexibility to the individuals appearing before them and more responsive to the requests for continuances or other needs.

To this end, we will train Observers (law students and others) to attend these public hearings and take notes about the procedures used. Observation may be in person or remote through the WEBEX viewer.

The procedure will be to randomize Observers to hearings, and then compare the procedural choices made in observed hearings (treatment) with those made in hearings not observed (controls). In addition, we want to know whether any difference in procedural choice might be due only to the judge’s heightened concern about making “improper” choices while being audited, or whether the judge might also respond to the judges’ supervisor knowledge about the particular judge rendering “improper” decisions while being audited. The EOIR requires the judges and their support staff to code each procedural choice in the individual hearings. This data is released pursuant to standing FOIA requests on the EOIR website. We will be focusing on the observations in three New York City area immigration courts which is one of the largest adjudication workloads in the national system.

The number of treated hearings will be determined based on the available budget. We expect an Observer-day to cost about $75, and an Observer audits one hearing a day. EOIR procedures divide regular removal hearings into status dockets called "Master Calendars" and merits hearings called "Individual Hearings." Master Calendars vary in size but average between 30 and 50 individuals in a four hour sitting. Our observations are only of Master Calendars. In these hearings (or dockets), the judge may take a wide variety of actions primarily related to scheduling or setting deadlines for applications such as asylum relief. Cases can be concluded during a Master Calendar hearing.

Given the available budget ($20,000), we expect to be able to “treat” about 266 hearings. The number of control hearings will exceed the number of treated hearings.

Intervention

The experimental subjects are approximately 70 judges in the New York City immigration courts. The goal of the research is to assess whether these judges respond to being observed by law students or other trained observers and, in particular, whether being observed makes a judge responsive to requests for more time for the individual to find legal representation (not provided at government expense in this forum) or to prepare applications for relief. To this end, we will train Observers (law students and others) to attend these public hearings and take notes about the procedures used. Observation may be in person or remote through the WEBEX viewer.
The procedure will be to randomize Observers to hearings, and then compare the procedural choices made in observed hearings (treatment) with those made in hearings not observed (controls). In addition, we want to know whether any difference in procedural choice might be due only to the judge’s heightened concern about making “improper” choices while being audited, or whether the judge might also respond to the judges’ supervisor knowledge about the particular judge rendering “improper” decisions while being audited.
There are three groups:
1. Judges who are audited in a particular hearing
2. Judges who are audited in a particular hearing and told that the auditor’s presence will be reported to the judge’s supervisor
3. Control group: judges who are not audited on that particular hearing

Procedural choices on all immigration hearings (outcomes on both treated and controls) are made publicly available by the DOJ through their EOIR website. The EOIR requires the judges and their support staff to code each procedural choice in the individual hearings. This data is released pursuant to standing FOIA requests on the EOIR website. We will be focusing on the observations in three New York City area immigration courts which is one of the largest adjudication workloads in the national system.

Experimental design

The experimental subjects are approximately 70 judges in the New York City immigration courts. The goal of the research is to assess whether these judges respond to being observed by law students or other trained observers and, in particular, whether being observed makes a judge more motivated to provide flexibility to the individuals appearing before them and more responsive to the requests for continuances or other needs. To this end, we will train Observers (law students and others) to attend these public hearings and take notes about the procedures used. Observation may be in person or remote through the WEBEX viewer.
The procedure will be to randomize Observers to hearings, and then compare the procedural choices made in observed hearings (treatment) with those made in hearings not observed (controls). In addition, we want to know whether any difference in procedural choice might be due only to the judge’s heightened concern about making “improper” choices while being audited, or whether the judge might also respond to the judges’ supervisor knowledge about the particular judge rendering “improper” decisions while being audited.
There are three groups:
1. Judges who are audited in a particular hearing
2. Judges who are audited in a particular hearing and told that the auditor’s presence will be reported to the judge’s supervisor
3. Control group: judges who are not audited on that particular hearing

Procedural choices on all immigration hearings (outcomes on both treated and controls) are made publicly available by the DOJ through their EOIR website. The EOIR requires the judges and their support staff to code each procedural choice in the individual hearings. This data is released pursuant to standing FOIA requests on the EOIR website.

Design details

Randomization method
We will randomly assign auditors to judges using a web interface that, for a given day in which the auditor declares his/her availability, randomizes the auditor to a hearing among the judges who hold a hearing on that day.
Randomization unit
The randomization unit is at the judge-hearing level.
Clusters
The number of treated hearings will be determined based on the available budget. We expect an Observer-day to cost about $75, and an Observer audits one hearing a day. Given the available budget ($20,000), we expect to be able to “treat” about 266 hearings. EOIR procedures divide regular removal hearings into status dockets called "Master Calendars"" and merits hearings called "Individual Hearings." Master Calendars vary in size but average between 30 and 50 individuals in a four hour sitting. Our observations are only of Master Calendars. In these sittings or dockets, the judge may take a wide variety of actions primarily related to scheduling or setting deadlines for applications such as asylum relief. Cases can be concluded during a Master Calendar hearing. The number of control hearings will exceed the number of treated hearings. We will cluster at the judge-hearing level, both for treated and for control hearings.
Observations
Each hearing usually involves 30-50 cases
Arms
Target number of treated hearings: about 266. Target number of control hearings: as many as available from the EOIR database during the sampling period. 1. 133 hearings of judges who are audited 2. 133 hearings of judges who are audited and told that the auditor’s presence will be reported to the judge’s supervisor 3. As many as possible judged in the control group: judges who are not audited on that particular hearing
Primary outcomes
We will collect data both at the hearing level and regarding the future evolution of audited cases, as they unfold. These outcomes will be recorded from the EOIR administrative data. Outcomes at hearing level Whether the audited hearing is 1. the last hearing 2. adjourned 3. if adjourned, different adjournment reasons Outcomes at the case level 1. Case duration 2. Number of hearings 3. Appeal rate 4. Decision type (e.g., administrative closure, remove, etc.)
Secondary outcomes
1. Length of the hearing

Cited by

No citation data available for this registration yet.

Keywords