Leveling the Playing Field for High School Choice: A Field Experiment of Informational Interventions

How to cite

Corcoran, Sean; Jennings, Jennifer; Sattin-Bajaj, Carolyn; Cohodes, Sarah. 2018. "Leveling the Playing Field for High School Choice: A Field Experiment of Informational Interventions." AEA RCT Registry. AEARCTR-0002951. https://doi.org/10.1257/rct.2951-5.199999999999999

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sean Corcoran* , Jennifer Jennings , Carolyn Sattin-Bajaj , Sarah Cohodes

* Primary investigator

Registry ID
AEARCTR-0002951
DOI
https://doi.org/10.1257/rct.2951-5.199999999999999
First registered
Last updated
Status
Completed
Trial dates
2014-08-04 – 2016-08-09
Countries
United States of America (New York City)
JEL codes
D83, I21, I24
Registry record
http://www.socialscienceregistry.org/trials/2951

Abstract

We conducted a school-level randomized controlled trial in 165 high-poverty New York City middle schools to help 8th grade students navigate a complex high school choice process and access higher-performing schools. The trial included 19,109 students across all study schools. Students in treatment schools were given a customized one-page list of proximate high schools with a graduation rate at or above the city median (70%). Some also received a supplemental list highlighting academically non-selective schools or high schools organized by academic interest area. The interventions changed student application behavior in ways that led to more matches to higher-performing schools. While treatment students did not apply to higher graduation rate schools, they applied to schools where their odds of admission were higher, were more likely to receive their first-choice high school, and were less likely to match to a school with a low graduation rate. Our findings also suggest that informational interventions may not reduce inequality, since both disadvantaged and comparatively advantaged students used our materials, and in some cases the latter benefited more from them by applying and matching to more schools on our lists. Future work will continue to follow these students and track their academic outcomes.

Intervention

Informed by the existing literature and our understanding of the challenges faced by NYC students choosing high schools, we developed a one-page informational tool called “FastFacts,” a customized list of 30 high schools for each middle school in our study. We designed Fast Facts to be an accessible starting point for students, and a useful reference for school performance information and admissions requirements. The intent was not for students to limit their search to these schools, but rather to begin with a initially smaller set of choices, and to be more aware of higher-performing schools in their proximity.

Schools recruited to the study were randomized into three treatment groups and a control group. Eighth grade students in the first treatment arm (FF1) received the basic Fast Facts school list described above. Students in the second treatment arm (FF2) received Fast Facts and a supplementary list of academically non-selective schools that give priority admission to students who attend an open house or information session and sign in. This group could also opt in to receive weekly text message reminders about open house dates, time, and locations. Schools in the third treatment arm (FF3) received Fast Facts and a supplementary list of high school programs organized by academic interest area. Both supplemental lists were attached to Fast Facts, on the inside of a bi-folded sheet. All treatment schools received a separate one-page insert of “screened language” programs citywide that serve recent immigrants and students learning English.

Materials specific to each intervention were delivered direct to students by trained research assistants via a 40-minute standardized lesson in a group setting (often but not always in a classroom). This lesson explained how to use the tools and emphasized the importance of graduation rates, admissions methods, and location when making school choices. All materials--including text messages--were available in English and Spanish, and lessons were delivered in Spanish when requested by the school guidance counselor. Control schools did not receive any materials until after the study was complete. (These schools were later provided Fast Facts lists that they could used with their 7th grade students).

Experimental design

Schools were assigned to one of the three “Fast Facts” interventions described in the “interventions” section, or a control group. We recruited 165 schools from the more than 500 schools serving 8th grade students in NYC, focusing on some of the highest-poverty schools in the city. To increase power over a simple cluster randomized trial in which schools are randomly assigned to treatment conditions, we randomized schools within 39 blocks of similar schools located in the same borough. Blocks were matched quadruplets of schools selected using a Mahalanobis distance measure of difference between schools (see Bruhn & McKenzie 2009; King et al. 2007). School variables used in the matching procedure included prior choice outcomes (e.g., the mean graduation rate of first round matches in 2013-14), prior achievement (mean ELA and math scores in 2013-14), economic disadvantage (the percent eligible for free or reduced price meals), and school size. The 23 schools in our pilot study were blocked separately. For details on randomization please see the online appendix to our NBER Working Paper available here: http://www.nber.org/data-appendix/w24471/w24471.appendix.pdf

Design details

Randomization method
computer program
Randomization unit
school
Clusters
165 schools
Observations
19,109 students
Arms
39 schools to Fast Facts 1, 39 schools to Fast Facts 2, 40 schools to Fast Facts 3, and 47 schools to control
Primary outcomes
a. High school choice outcomes (for 1st, top 3, and all choices; matched school; and enrolled school) i. School presence on the Fast Facts and/or supplementary lists 1. presence on the list 2. presence in the top or bottom half of the list ii. High school characteristics 1. 4-year graduation rate 2. whether the graduation rate was below the Fast Facts threshold of 70% 3. travel time from the middle school to the high school 4. location in the same borough as the student’s residence 5. applications per seat in the prior year (a measure of demand) 6. admissions method (e.g. screened or limited unscreened, different types of high schools in the NYC system) 7. variability in graduation rates (the difference between the highest and lowest graduation rate of schools appearing on a student's application) 8. academic or career interest area (e.g. STEM) 9. within-application consistency in interest area, calculated as the highest percentage of choices from the same interest area i. Other admissions process and enrollment outcomes 1. number of choices submitted (up to12) 2. open house priority status for limited unscreened programs (a lottery preference given to students who attend an event for certain schools) 3. ranking of the student by screened programs 4. whether or not the student was matched to his or her 1st choice, 1st-3rd choice, or any choice in the first round 5. participation in the second round after a successful match 6. matriculation to the matched school in 9th grade. Other outcome variables b. Students’ academic progress and other outcomes in high school i. Credit accumulation in 9th-12th grade, and GPA; ii. New York State Regents exam scores and passing rates; iii. On-time graduation; iv. Engagement and perceptions of school (e.g. attendance rate and responses to the NYC DOE’s Learning Environment Survey) C. Post-high school outcomes (pending data availability) i. College enrollment, persistence, and graduation ii. Earnings iii. Arrest records iv. Voter registration, voting

Papers from this trial

  1. Sean P. Corcoran, Jennifer L. Jennings, Sarah R. Cohodes, and Carolyn Sattin-Bajaj. 2018. Leveling the Playing Field for High School Choice: Results from a Field Experiment of Informational Interventions. NBER Working Paper No. 24471. https://www.nber.org/papers/w24471

Cited by

No citation data available for this registration yet.

Keywords