The Effects of Student Growth Data on School District Choice: Evidence From a Survey Experiment

How to cite

Houston, David; Henig, Jeffrey. 2018. "The Effects of Student Growth Data on School District Choice: Evidence From a Survey Experiment." AEA RCT Registry. AEARCTR-0003401. https://doi.org/10.1257/rct.3401-4.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

David Houston* , Jeffrey Henig

* Primary investigator

Registry ID
AEARCTR-0003401
DOI
https://doi.org/10.1257/rct.3401-4.0
First registered
Last updated
Status
On going
Trial dates
2018-10-17 – 2019-06-15
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/3401

Abstract

We seek to identify the effects of providing district-level average student achievement data and/or average student growth data on subjects’ hypothetical school district enrollment decisions. Compared to average achievement at a single point in time, average growth is arguably better able to capture schools’ and school systems’ contributions to student learning. Average growth also bears a weaker relationship to the racial and socio-economic composition of the student body, making it easier to identify highly effective schools that serve less advantaged students.

This study consists of an online survey experiment in which each subject is asked to imagine that s/he is a parent who is moving to a new city. When deciding where to live, one of their top priorities is to choose a school district for their elementary school-age child. The survey will provide basic demographic information about the five largest school districts in the metropolitan area. In addition to the demographic information, subjects will be randomly assigned to receive either 1) average achievement data, 2) average growth data, 3) both, or 4) neither. Based on these data, subjects will choose their preferred school district.

Subjects will be asked to complete this process for the metropolitan areas of the five largest cities in the United States: New York City, Los Angeles, Chicago, Houston, and Phoenix. At the end of the survey, subjects will answer a small battery of basic demographic questions.

Intervention

This study consists of an online survey experiment in which each subject is asked to imagine that s/he is a parent who is moving to a new city. When deciding where to live, one of their top priorities is to choose a school district for their elementary school-age child. The survey will provide basic demographic information about the five largest school districts in the metropolitan area. In addition to the demographic information, subjects will be randomly assigned to receive either 1) average achievement data, 2) average growth data, 3) both, or 4) neither. Based on these data, subjects will choose their preferred school district.

Subjects will be asked to complete this process for the metropolitan areas of the five largest cities in the United States: New York City, Los Angeles, Chicago, Houston, and Phoenix. At the end of the survey, subjects will answer a small battery of basic demographic questions.

Please see pre-analysis plan for more detail (also available at https://osf.io/3e8wv/)

Experimental design

We are planning on conducting an online survey experiment with four experimental conditions. For the five largest school districts in the metropolitan areas of the five largest US cities, subjects will be randomly assigned to receive either 1) average achievement data, 2) average growth data, 3) both, or 4) neither. The estimands will be the average differences in the racial and socio-economic compositions of the school district choices between experimental groups.

Please see pre-analysis plan for more details (also available at https://osf.io/3e8wv/)

Design details

Randomization method
We will use the Qualtrics survey platform to administer the survey experiment. Using Qualtrics’ “Randomizer” feature, roughly equal portions of the subject pool will be randomly assigned to the control group (receiving neither achievement nor growth data), the achievement group (receiving only achievement data), the growth group (receiving only growth data), or the combination group (receiving both achievement and growth data).
Randomization unit
Randomization occurs at the individual level. For a few summary analyses, we will reorganize the data into a "long-form" dataset (each individual will appear in the data five times: once for each metropolitan area school district choice). When analyzing these data, we will cluster standard errors at the subject level.
Clusters
2,500 individuals
Observations
2,500 individuals
Arms
625 control, 625 achievement group, 625 growth group, 625 combination group
Primary outcomes
For each metropolitan area, there will be three outcome measures: subjects’ choices with respect to 1) median household income, 2) the percentage of students eligible for free and reduced priced lunch, and 3) the percentage of white students. The estimands for each metropolitan area will be the average differences in the racial and socio-economic compositions of the school district choices between experimental groups. To estimate average treatment effects across all five metropolitan areas, we will reorganize the data into a long-form dataset. Each subject will appear in the dataset five times: once for each school district selection. The estimands for these data will also be the average differences in the racial and socio-economic compositions of the school district choices between experimental groups. When conducting these analyses, we will cluster standard errors at the subject level.

Cited by

No citation data available for this registration yet.

Keywords