The Effects of Legal Assistance on Evictions
How to cite
Rafkin, Charlie; Caspi, Aviv. 2022. "The Effects of Legal Assistance on Evictions." AEA RCT Registry. AEARCTR-0010687. https://doi.org/10.1257/rct.10687-1.3
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We conduct a randomized controlled trial (RCT) of the effect of providing attorneys to tenants facing eviction. We partner with a local nonprofit, Neighborhood Preservation, Inc., in Memphis, Tennessee. NPI has received independent grants to provide 200–600 lawyers to represent tenants with eviction filings in Shelby County General Sessions Court. We assist NPI with randomizing the provision of lawyers. We study the impact of lawyers on three groups of outcomes: (1) formal eviction outcomes, including judgments, writs, nonsuits, time in court, and money owed to the landlord; (2) informal eviction outcomes collected via an endline survey, including moves, and informal bargaining; (3) financial outcomes collected via the endline survey and merged credit reports. We also study a fourth group of outcomes: (4) other outcomes collected in baseline surveys (e.g. beliefs, willingness to pay), though we do not study the treatment effects of lawyers on these outcomes since they are collected prior to treatment.
Disclaimer: The trial is registered in a staggered fashion. In v1.0, we registered outcomes in Group 1 and the primary treatment. We also gave our best prediction about registration of the secondary treatment and outcomes in Groups 2-3. In v1.1, we registered outcomes in Group 2, prior to launching the endline surveys. In v1.2, we register outcomes in Group 4, prior to launching the baseline surveys. In v1.3, we make a few light changes: (a) updating few aspects of the registration to Group 4 based on feedback (within one month of launching the baseline survey, N < 30 collected), (b) adding some secondary heterogeneity, and (c) adding plans to test for attrition.
In all registrations, we provide information based on our present understanding. The partner is sponsoring the provision of attorneys and we cannot delay the treatments while we get more information or pilot.
We emphasize that the outcomes in Group 2 and Group 4 are fully preregistered, except where minor revisions are indicated. The respective surveys had not been launched at the time of their initial registration. We were able to observe some preliminary results from Group 1 with a pilot sample that we intend to pool with the main estimates. That is why we registered these outcomes as soon as we could in v1.0.
Timeline:
v1.0 - initial registration, after treatment assignment for some participants and after some administrative outcomes collected. Before any surveys collected and credit reports purchased.
v1.1 - updated registration for outcomes collected in endline survey. Fully preregistered before endline surveys launched.
v1.2 - updated registration for outcomes collected in baseline survey. Fully preregistered before baseline survey launched. We also clarify that the control group, when they receive a notice that they were not selected for representation, also receives some information. This has been a part of the study since before registration in 1.0, and nothing has changed about the treatment. But we wanted to make it clearer in the registry.
v1.3 - updated registration to: include additional heterogeneity in the baseline survey; change plans for how we will use reference good; propose plans to study attrition using several sub-randomizations; update plans for secondary treatment; and register that our measures of trust are elicited via Trust Games.
For primary outcomes:
Group 1 was registered after data collection began (v1.0).
Group 2 was preregistered before endline data collection began (preliminary in v1.0, registered v1.1).
Group 3 remains preliminary (through v1.2).
Group 4 was preregistered before baseline survey data collection began (preliminary through v1.1, registered in v1.2, updates in v1.3).
Note: We pre-register secondary survey outcomes before data is collected. Some secondary survey outcomes are in Group 3 and are indicated as such in the secondary outcomes section.
Intervention
There are two treatments and a control group.
The primary treatment group provides full legal assistance to tenants facing eviction. In particular, tenants who are randomized into this treatment will receive an offer for full legal representation in General Session court. The legal team constitutes a mix of experienced tenant lawyers and lawyers whom NPI recruited and trained to conduct “low-bono” legal work.
The secondary intervention provides light-touch (one-time) legal counseling and social work to tenants facing eviction. In particular, tenants in this group receive a one-time phone call from either law students or a social worker. The law students and social workers are trained by NPI to provide coaching about how to handle an eviction. We registered the intervention as “secondary” starting in v1.0, because faced uncertainty about whether NPI will hire enough social workers or law students to reach a large number of tenants. As of v1.3, it became clear that the nonprofit’s social workers will not reach a large share of the control group, and the ones they do reach may be called after the eviction proceedings. We continue to randomize social workers as a part of our impact evaluation/assistance we provide NPI. We may include this treatment in the paper as a secondary treatment. But we do not currently plan for it to be a focus.
The control group does not receive assistance. When they are informed via email that they were not selected for assistance, the email notification provides select other resources to which they can apply in Memphis, as well as some basic information about their rights in eviction court. This information is also provided to the attorneys and counselors as part of their training. Prior to v1.2, we had labeled this group as a “pure control,” because we view this information as light-touch and unlikely to make a large difference in tenant outcomes. Nothing about the control group has changed — just how we label it.
Given that there are two treatments, we discuss our intended comparisons below in the experimental design section.
Experimental design
Design and intake. Tenants apply for the program at NPI’s website. Upon completion of the application, tenants are immediately directed to our baseline survey (more details below). Tenants are presently eligible to receive full legal assistance only if they already have received an eviction filing. After determining eligibility, tenants are randomized into receiving an offer for full assistance.
Tenants are eligible to receive one-time counseling even if they have not received an eviction filing. We randomly select some tenants to enter a queue for one-time counseling.
Intended comparisons. Because we have two treatments and a control, we preregister our “primary” and “secondary” comparisons.
We have uncertainty about the number of people who will be treated with the light-touch (secondary) treatment. This affects what we anticipate the primary comparisons in the experiment will be.
If that secondary treatment remains small relative to the control, then our main comparison will be between the primary treatment and the control. If the secondary treatment are a larger share of the sample, then our main comparison will be between the primary versus versus the secondary treatment pooled with the control.
ITT/IV. We will study the Intent to Treat effect of being offered a lawyer. There is incomplete compliance since some tenants are no longer eligible, do not reply to, or do not accept the offer of a lawyer (even though they must first apply for one). We will also use the random variation in the offer to instrument for the effect of lawyers on a given outcome.
Timing of outcomes. We will present results at sensible timeframes (e.g., 3, 6, and 12 months after filing or application). We will also present survival curves, in which we will test hypotheses using Wilcoxon tests or similar.
Randomization. NPI began providing the primary legal assistance treatment starting in March 2022. NPI asks us to randomize tenants into the sample based on the availability of the legal counsel. Because we are merely assisting the partner with randomization on a program that they intend to do otherwise, we do not have scope to stop the sample for piloting.
Pooling with Pilot sample. We have been piloting the process since March 2022 and initially registered in December 2022. There was (and remains) uncertainty about how large the secondary treatment will be and how large the sample will be. As a result, we have delayed preregistration until we had more information. Since the uncertainty has not yet been resolved, in the interest of transparency, we preregister our outcomes in December 2022 with the intention of pooling pilots with the main sample wherever possible, as our primary specification. We will also show all our estimates in separate exhibits that drop the pilot sample (who applied before the date of the preregistration), but we do not presently anticipate that these dropped estimates will be our main sample.
The only rationale we foresee that may change the assessment of what our main sample will be is the interaction with the local Emergency Rental and Utilities Assistance Program, as explained above. If it turns out that the access to funds from the local ERAP was critical and the program had very different levels of effectiveness when ERAP was available, that could motivate separating the pilots that had access to the local ERAP from the primary analysis sample.
Endline surveys and collection of informal outcomes. We intend to field phone or online surveys of tenants to measure the informal outcomes listed above. Separate from these surveys, lawyers or counselors can record many informal outcomes for treated individuals. There is a natural concern about differential attrition between the treated and control sample. To address this concern, we intend to field the survey among the treated sample as well. That will permit us to test for and/or adjust the estimates by comparing the treated group’s survey outcomes to the “ground truth” recorded by lawyers. The survey outcomes will also let us study whether the tenant is employed and their monthly income from employment.
We note that there are both baseline surveys and endline surveys. Treatment assignment occurs in a staggered fashion over more than a year. We launched the endline surveys on January 26, 2023 (v1.1). We launch the baseline surveys on March 2, 2023 (v1.2); they will clearly not cover all participants.
Links to financial data. Credit-report outcomes in outcomes in Group 3 above are preliminary. We will update this part of the preregistration when we have more information about whether these linkages are feasible. Broadly, we intend to purchase credit report credits from one of the three major credit companies in the United States and link them to the treatment and control group. We will examine standard outcomes in the literature, especially the ones emphasized in Collinson et al. (2022) for comparability.
Multiple hypothesis testing. We will appropriately adjust for multiple hypotheses within groups of outcomes above.
Baseline survey. After applying for the program, tenants are redirected to an online portal where they can take the baseline survey. Depending on response rates, we may additionally employ a surveyor to call tenants to take the survey on the phone. In the baseline survey, we conduct the following modules: tenants’ willingness to pay for lawyers, counselors, and a reference good (an iPad); beliefs about the share of tenants who receive eviction judgments (if there is a filing), if the tenants do and do not have lawyers; standard elicitations about credit constraints; information about the relationship between the tenant and landlord; whether the landlord or property manager recently changed; whether the tenant intends to go to court for their case; information about past interactions between the tenant and lawyers; and measures of trust that the tenant has for various occupations in society (measured via a Trust Game) as well as general trust in others (qualitative question).
These outcomes are collected pre-treatment. They serve as: (i) sources of heterogeneity; (ii) independently useful/interesting parameters for welfare calculations; (iii) comparisons to non-applicants when studying questions relating to selection and take-up; (iv) experimental tests of the effect of budget constraints on WTP for lawyers and counselors. We collect a generalized measure of attention in the baseline survey (“Select the number two”) and will drop tenants who fail this attention check. In our primary analysis, we will not use the pilot sample, but we may aggregate the primary sample with the ~25 pilot participants for power.
We now describe several details about the willingness to pay elicitation. These are conducted as multiple price lists, incentivized using a Becker-Degroot-Marshak mechanism. Each tenant has three WTPs elicited: for a lawyer, a counselor, and a reference good (an iPad). Tenants are randomized into reporting WTPs when we give them only $50 if they are selected (and then trade off more money versus the good), versus when we give them $500 if they are selected. The purpose of this elicitation is to test if materially relaxing budget constraints affects WTP. As a secondary outcome, we test the effect of relaxing the budget constraint on the reported WTP.
We also collect a hypothetical WTP to have an eviction delayed, to measure the potential effect on tenant well-being from court delays. We randomize the number of weeks that the eviction would be delayed, which allows us to trace a demand curve. This elicitation is hypothetical, so it is registered as secondary.
Attrition tests [added v1.3]. An important concern is that the endline surveys we conduct will have low take-up rates. We randomize whether we reach out to the tenant via a phone call from a surveyor or email (and only called if the surveyors have sufficient capacity). Among a subset of emailed tenants, we also randomize the payment ($8 or $15). We will use these randomizations as instruments for participation to study the effect of attrition.
References
Collinson, Robert, John Eric Humphries, Nicholas S. Mader, Davin K. Reed, Daniel I. Tannenbaum, and Winnie Van Dijk. “Eviction and Poverty in American Cities.” 2022.
Dizon-Ross, Rebecca, and Seema Jayachandran. "Improving Willingness-to-Pay Elicitation by Including a Benchmark Good." In AEA Papers and Proceedings, vol. 112, pp. 551-555. 2022.
Design details
Cited by
No citation data available for this registration yet.