Identifying The Spillover and Congestion Effects of Large Scale Information Interventions

How to cite

Neilson, Christopher; Gallego, Francisco; Cruz, Claudia Allende Santa. 2020. "Identifying The Spillover and Congestion Effects of Large Scale Information Interventions." AEA RCT Registry. AEARCTR-0006302. https://doi.org/10.1257/rct.6302-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Christopher Neilson* , Francisco Gallego , Claudia Allende Santa Cruz

* Primary investigator

Registry ID
AEARCTR-0006302
DOI
https://doi.org/10.1257/rct.6302-1.0
First registered
Last updated
Status
On going
Trial dates
2020-08-05 – 2021-03-31
Countries
Private
JEL codes
D83 Search-Learning-Information, I24 Education and Inequality, I28 Government Policy, L15 Information and Product Quality
Registry record
http://www.socialscienceregistry.org/trials/6302

Abstract

The main objective of this project is to study how the effects of information provision vary with the scale of the intervention. We evaluate the effects of an intervention that provides information of product availability and product characteristics on consumer demand. It is hypothesized that the potential changes to consumer demand can vary due to information spillovers when many consumers in the same market are exposed to the information treatment. Heterogeneous effects across types of consumers could increase or decrease differences in the allocation of products across consumers. On the other hand, if many consumers are more informed of their options and behave in a similar way, this may create congestion in the short run, leading some consumers to not be able to get what they wanted even if in the medium run, supply can adjust to accommodate the change in demand. We study these issues in the context of school choice where consumers are families that submit a ranked ordered list to a centralized assignment mechanism and are later assigned to schools depending on a specific set of rules, capacities, and overall demand. This allows for the design of a study where we can in theory disentangle the spillover effects of the information intervention on choices separately from the congestion effects that occur when aggregate demand changes prior to any supply side adjustments. We provide individual families as well as entire neighborhood markets with information about the schools that are nearby. We observe their applications as well as their later assignments and matriculation choices. We can test whether the intervention changed their applications as well as their later assignments. The eligible population for this study consists of all parents from dense urban areas (i.e. those with many schooling options) who participate in the centralized school choice process for the academic year 2021 in Chile. Since our implementing partner is the school choice administration itself, we are able to include almost the entire eligible population as part of our study. Our goal is to help determine where information provision at scale can be helpful at improving match quality and reducing segregation while also taking into account that a scaled-up intervention could have adverse effects on some applicants due to increased congestion in the short run.

Intervention

Control: Basic report card
Parents that belong to the control group will receive a report card with information regarding the results of the admission process in 2019 for schools in their neighborhood (i.e. schools located within 2 km of the reported home address) in addition to detailed information about the schools that parents considered in their application portfolio (this is the POTENTIAL portfolio for those who were recruited before the application period opened, or the ACTUAL portfolio for those who were recruited once they submitted an official application).

Treatment: Basic report card + Detailed report card on schools within 2km + Video
Parents that belong to the treatment group will receive the same report card as the control group plus a second report card that shows 1.) a map with all the schools within 2 km of the reported address of the household and 2.) detailed information of all the schools within 2 km of the household. This means that treated parents will most likely receive information about schools that they had NOT considered in their application portfolio, therefore expanding their consideration set. This group will also receive a video that explains the importance of being informed in order to apply to schools and the returns of getting a good education early in the students’ life.

Experimental design

The eligible population for our study are the parents of students applying to school in an entry grade (Pre-K, Kindergarten, First grade, Ninth grade), in Chile’s dense urban areas, for the academic year beginning in March 2021. As mentioned before, we will be able to include nearly the entire eligible population in our sample, because we are working with the school choice agency directly. In fact, for political restrictions, it is not possible to include only the mentioned grades in our study, and therefore in the implementation we will consider applicants from any grade (Pre-K through 12). However, in the analysis we will only consider those from entry grades. The majority of applicants belong to those entry grades, as expected. See the Pre-Analysis Plan for more details.

From the beginning of August 2020, the school choice portal will display a banner that reads “access personalized information here”. Parents who click on the banner will land on the page that is dedicated to this policy pilot (tuinformacion.mineduc.cl), where they are prompted to sign up. This constitutes the first channel of recruitment. By signing up, they become part of the RCT sample, provided they live inside our selected clusters. While signing up, we will request contact information (email, phone number for SMS and Whatsapp) and, importantly for the randomization, their home address. Once they are logged in, they will be asked to enter the list of all the schools they know and how they wish to rank them in their actual application. All of those who are recruited in this way constitute sample 1, and they will be randomized into treatment by clusters.

Additionally, anyone who completes a real application, will also be part of the experiment, provided they live inside our selected clusters. The difference with these applicants is that they enter the experiment at a moment when they have already submitted a real application. They will be randomized into treatment via clusters, and we will refer to this sample as sample 2.

Additionally, on day three of the application process, we will take all the students from sample 2 that fall inside the control clusters, and we will randomly select a subsample that will be randomized into treatment at the individual level. We refer to this subsample as sample 3. In order to keep the control clusters relatively clean, this sample can not be very large. In practice, this means that there are few “slots” for sample 3, so we will prioritize students that do not have a particularly high switching cost To select this sample, we will proceed as follows. First, we will restrict the universe of applicants to the ones whose decision process is especially interesting for the types of questions our study seeks to answer: applicants choosing schools for the first time (i.e. without a secured spot in their current school), with no siblings enrolled in any school, and with at least 5 schools in a 2 km radius. Then, we will assign a random number to each individual that meets the criteria in the first step, and rank them based on that number. Next, we will proceed to select individuals for sample 3. We will add observations one by one, following the rank. Observations will only be added to the sample if they meet the criteria of being separated by at least 0.5km from other individuals already in the sample.

In summary, the unit of randomization is the cluster for samples 1 and 2, or the parent, for sample 3. The clusters were constructed using the geographic distribution of applications in 2019, employing a Density-based spatial clustering of applications with noise (DBSCAN) algorithm.

Design details

Randomization method
Randomization is done in an office by a computer.
Randomization unit
For samples 1 and 2: the cluster For sample 3: the parent
Clusters
212 clusters (only relevant for samples 1 and 2. Sample 3 is randomized into treatment individually).
Observations
around 400,000 parents
Arms
Sample 1 - Treatment: 3,000 parents Sample 1 - Control : 3,000 parents Sample 2 - Treatment: 200,000 parents Sample 2 - Control: 200,000 parents Sample 3 is a subsample of Sample 2 and has to be selected during the implementation to meet the restrictions we imposed in the design. We estimte it will consist of about 2,000 parents.
Primary outcomes
At the individual level: Mean test scores of schools in application (Primary application outcome) Test scores of the school student attends in March 2021 (Primary placement outcome) At the cluster level: Probability of poor (SEP) student attending a school of higher tertile of quality in the cluster in March 2021 (Pimary distributional outcome)
Secondary outcomes
Mean value added of schools in application [secondary outcome] Mean price of schools in application [secondary outcome] Mean distance to schools in application [secondary outcome] Number of schools listed in application [secondary outcome] Change in application (this is applicable for parents that are recruited through the SAE directly, because they have already submitted an application at the moment we treat them.) [secondary outcome] Value added of the school they attend in March 2021 [secondary outcome] Price of the school they attend in March 2021 [secondary outcome] Distance to school they attend in March 2021 [secondary outcome] Average number of applications received by schools that belong to the higher within-cluster tertile of quality (school-level outcome) [secondary outcome]

Cited by

No citation data available for this registration yet.

Keywords