Improving the Quality of Legal Aid: Impact Evaluation of Tech-Enabled Mediation in Peru

How to cite

Maqueda, Manuel Ramos; Chen, Daniel Li; Silveira, Bernardo Santos da; Kim, Galileu. 2021. "Improving the Quality of Legal Aid: Impact Evaluation of Tech-Enabled Mediation in Peru." AEA RCT Registry. AEARCTR-0008607. https://doi.org/10.1257/rct.8607-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Manuel Ramos Maqueda* , Daniel Li Chen , Bernardo Santos da Silveira , Galileu Kim

* Primary investigator

Registry ID
AEARCTR-0008607
DOI
https://doi.org/10.1257/rct.8607-1.0
First registered
Last updated
Status
In development
Trial dates
2021-11-12 – 2023-02-01
Countries
Private
JEL codes
C01, C21, C93, D02, D63, D73, D74, D78, D91
Registry record
http://www.socialscienceregistry.org/trials/8607

Abstract

How to raise the quality of publicly provided services in in the face of tight resource constraints? In this project, we evaluate the introduction of a tech-based, low-cost platform allowing extra-judicial mediators in Peruvian free mediation centers to monitor their own performance. We assess the impact of the platform on both mediators' performance and on socio-economic characteristics of the users of these mediation services.

Intervention

Partnering with the Peruvian Ministry of Justice, we developed a platform named "App del Conciliador" (Mediator App), which leverages rich administrative data on extra-judicial mediation cases from Peruvian free public mediation centers. The platform provides mediators with rolling performance reviews that we expect to increase the salience of outlier characteristics, such as unusually lengthy average resolution times compared with colleagues. We aim to make mediators aware of their own abilities and the key areas for improvement. Thus, mediators may become better self-managers and can proactively address shortcomings, while prioritizing casework that optimizes the marginal value of their limited time. In addition, by comparing their performance to that of other peer mediators on similar case types, the Mediator App will provide feedback on what areas they have greater room to improve on.

Experimental design

In our experiment we use simple lottery to assign mediation centers either to the control group or to our single treatment consisting in access to the Mediator App. The mediators belonging to mediation centers selected for treatment (50%) will receive access to the App and weekly reminders to log in. In the App, they will be able to monitor their performance on three key indicators: the number of mediation cases they work on, the rate of settlements with partial or full agreement, and the average number of days it takes them to close a mediation file. Their current performance will be compared to both their past performance and the average performance of all mediators. This will provide a clear indication to each mediator of the level of his/her own performance on each of these metrics, and demonstrate any progress or setbacks.

Design details

Randomization method
First, we retrieved from the administrative data all the mediators who were at any point active in the dataset. We combined this data with a short-list of all on duty mediators provided to us by the Ministry of Justice to have the universe of all on-duty and off-duty (but potentially returning) mediators. We proceed with stratified randomization for treatment assignment for mediators belonging to a single-mediator center. The strata are defined as: 1) the number of cases mediated by the mediator, 2) district population and 3) on/off-duty status. We convert the first two variables into binary variables (high/low) to increase the size of each group and reduce imbalance between treatment and control. The probability weight for treatment assignment was 0.5. Additionally, 8.9 percent of our sample of mediators belong to centers with multiple mediators - the remainder are centers with a single mediator. For those centers with multiple mediators, we randomly assigned them a treatment status with a 0.5 probability weight. In our randomization procedure, all centers with multiple mediators were assigned a treatment status. As a result, all mediators belonging to these centers were assigned to the treatment group. Randomization of treatment assignment leaves us with with 39 mediators in control and 40 mediators in the treatment group. To check for covariate balance, we estimate a simple OLS regression, using as outcome the treatment assignment, and as predictors a set of covariates that we consider as potentially correlated with potential outcomes: the department in which the mediator is located, population size of district, work experience, case duration and success rate for agreements. We find that almost all of the coefficients are not statistically significant and the covariates across treated and control groups are indeed balanced, with the exception of two departments out of 25 – Huanuco and Piura. To account for this minor imbalance, a full set of controls, including department fixed effects, will be included in our main model specificationss.
Randomization unit
mediation center level (where most centers have 1 mediator per center)
Clusters
75 mediation centers
Observations
For 14 weeks, we will observe the performance of 79 mediators every week.
Arms
treatment (i.e., access to app): 40 mediators from 39 mediation centers control: 39 mediators from 36 mediation centers
Primary outcomes
(A) performance-related self-awareness (B) performance (B.i) number of mediation cases (B.ii) percentage of successful mediation cases (B.iii) duration of mediation cases
Secondary outcomes
(I) economic status (II) health status (III) educational performance (IV) workload of the courts in the respective districts of free mediation centers

Cited by

No citation data available for this registration yet.

Keywords