Technology, Information, and School Choice: A Randomized Controlled Trial
How to cite
Cohodes, Sarah; Corcoran, Sean; Jennings, Jennifer; Sattin-Bajaj, Carolyn. 2019. "Technology, Information, and School Choice: A Randomized Controlled Trial." AEA RCT Registry. AEARCTR-0003736. https://doi.org/10.1257/rct.3736-2.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
New York City requires all 8th grade students to engage in a school choice process to be assigned to a high school through a centralized assignment mechanism. The admissions process is complex and cognitively demanding, requiring students to select among schools with different screening methods, curricular themes, and admissions preferences. To test the potential of information to help students navigate the process, we fielded a series of informational interventions in a school-level randomized controlled trial. The initial year of our study tested the effect of a paper-based informational tool – a list of relatively high-performing high schools -- in a school-level RCT of 165 high-poverty New York City middle schools (Corcoran, Jennings, Cohodes, and Sattin-Bajaj, 2018). We found that access to the informational tool increased applications to listed schools and decreased the likelihood that students attend a high school with a graduation rate below 70%.
We extended this project for an additional two years as an school-level RCT, which expanded the number of schools served to over 450 and the types of interventions offered to include a variety of technological innovations. We contrast a paper and a digital version of a curated list of schools, a guided search tool, and a directory search tool to a control group. School personnel received the intervention tools to distribute via mail, along with supportive materials (lesson plans, video guides, and support from the study office), simulating the most likely mode of dissemination if a school district were to adopt such tools. This was in contrast to the prior year’s study where trained research assistants delivered materials in a lesson. In 2016-2017, schools were randomly assigned to one of three informational interventions or a control group. In 2017-2018, we replicated the prior years random assignment to control or treatment and offered two informational interventions in the treated groups.
Intervention
Schools were assigned to one of three informational interventions or a control group. School personnel received the intervention tools to distribute via mail, along with supportive materials (lesson plans, video guides, and support from the study office), simulating the most likely mode of dissemination if a school district were to adopt such tools. Study personnel provided telephone support to encourage material use and answer questions. The three informational interventions were:
1) Curated List of High Schools ("Fast Facts"): this group received a middle school-specific concise listing of 26 geographically proximate high schools with graduation rates above 75%, along with travel time information, the graduation rate, and information about how to apply. In cross-randomization within this group, students received either a digital or paper version of the tool. A further cross-randomization made slight variations to the high school list. A third of high school lists included a caution about two nearby high schools that had low graduation rates and a third of the lists included a caution about two nearby high schools that had low admissions rates.
2) Personalized Information about High Schools from a School Choice “App”: this group received a guided introduction to an interactive smartphone/web-based tool designed to help students translate their preferences into a list of school recommendations. The app serves as a guided search tool, prompting students to identify their current middle school and their preferences for commute time, academic interests, and extra-curricular interests. It then generates a list of schools, along with performance data, that students can save, share, and explore further.
3) General Information about High Schools from a Searchable Directory ("School Finder"): This group received a guided introduction to the NYCDOE SchoolFinder, a search engine for finding high schools launched in the 2016-17 high school admissions cycle. Since all students had access to this tool (including in the control group), this group allows us to test the effect of the supportive materials that were offered as part of the intervention.
Experimental design
For 2016-2017:
Randomization was conducted in two “tiers.” Tier 1 was made up of schools that participated in our 2015-2016 RCT. Because school counselors may have provided prior years materials to students, and for ethical reasons, we decided that all schools in Tier 1 would receive a treatment, even if they were assigned to control in 2015-2016. Thus these schools contribute to estimating contrasts across treatments, but not comparisons to the control group. We retained the 39 blocks formed by matching in the previous year. See https://www.socialscienceregistry.org/trials/2951/history/37072 and http://www.nber.org/data-appendix/w24471/w24471.appendix.pdf for details on that randomization process. Within these 39 blocks, schools were randomly assigned to one of the three Fast Facts treatments, SchoolFinder, or the App. By block, the Fast Facts schools within the block were randomly assigned to either digital only delivery or digital and paper delivery.
Randomization for Tier 2 consisted of schools new to the study in 2016-2017, and were high and medium poverty schools in NYC. The approximately 100 lowest poverty schools in NYC were excluded from the experiment. Schools were randomized regardless of their intent to participate; that is, we did not recruit schools in advance for participation. We then blocked these schools into blocks of 6 schools (were possible). With 6 schools in a block, the modal block assigned one school to each of the three Fast Facts versions, one school to School Finder, one to the App, and one to control. Blocks were thus matched sextuplets of schools selected using a Mahalanobis distance measure of difference between schools (see Bruhn & McKenzie 2009; King et al. 2007). School variables used in the matching procedure included prior choice outcomes (e.g., the mean graduation rate of first round matches in 2015-16), prior achievement (mean ELA and math scores in 2015-16), economic disadvantage (the percent of students in poverty), and school size. To maintain face validity, blocking was conducted within borough and geographically isolated schools were blocked together (i.e. the Rockaways and Staten Island). Additionally, schools were blocked within categories based on their response to recruitment for the 2015-2016 experiment so that blocks were formed within groups of schools that had similar characteristics (e.g. school has returning 8th graders, school did not choose to participate in 2015-2016, school new to study, etc.) Within these blocks, schools were randomly assigned to treatments or control. The blocks were then listed in a random order and a cross randomization that alternated Fast Facts delivery method (digital only or digital and paper) was implemented.
For 2017-2018:
In the second year of the study, we used the extant randomization described above for 2017-2018. Schools that were previously assigned to one of the Fast Facts interventions were assigned to Fast Facts again (the low-graduation version) and schools that were assigned either School Finder or the App were assigned the App. Materials were “refreshed” and delivered electronically to guidance counselors. If a school received a new counselor since the previous year’s treatment, they were sent additional paper materials.
Design details
Papers from this trial
- Cohodes, Sarah R., Sean P. Corcoran, Jennifer L. Jennings, and Carolyn Sattin-Bajaj.. “When Do Informational Interventions Work? Experimental Evidence from New York City High School Choice.” National Bureau of Economic Research Working Paper 29690, 2022. https://www.nber.org/papers/w29690
Cited by
No citation data available for this registration yet.