How Information Affects Parents' Choice of Schools: A Factorial Experiment

How to cite

Glazerman, Steven; Nichols-Barrer, Ira; Valant, Jon. 2016. "How Information Affects Parents' Choice of Schools: A Factorial Experiment." AEA RCT Registry. AEARCTR-0001190. https://doi.org/10.1257/rct.1190-3.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Steven Glazerman* , Ira Nichols-Barrer , Jon Valant

* Primary investigator

Registry ID
AEARCTR-0001190
DOI
https://doi.org/10.1257/rct.1190-3.1
First registered
Last updated
Status
Completed
Trial dates
2016-08-01 – 2017-09-29
Countries
United States of America ()
JEL codes
I20, M38, R28
Registry record
http://www.socialscienceregistry.org/trials/1190

Abstract

School choice can only be an effective policy if choosers can process large amounts of information about schools to make effective choices. This study seeks to identify the impacts of different strategies of presenting consumers with information about schools on the choosers' ability to understand and use the information. We categorize school information into four domains: convenience (primarily distance from home), academics (primarily captured by academic proficiency and growth measures), safety (captured by indicators such as school suspension rates and parent perceptions of safety), and resources (captured by number of laptops or devices per student).

The study is an online experiment with 72 treatment arms arranged in a 3 x 3 x 2 x 2 x 2 factorial design. The study will ask respondents, who are screened to be low-income parents of school-aged children, to rank their top 5 among 16 hypothetical schools with detailed profiles. We will experimentally vary:
1. the format (numbers, numbers + graphs, or numbers + icons),
2. the source of information (objective indicators or objective + subjective indicators),
3. the presence or absence of a reference point, namely the district-level mean value for each indicator
4. the number of attributes per domain and disclosure method (one attribute per information domain, multiple attributes per information domain, or multiple attributes with progressive disclosure via user-initiated click-through to see beyond the first attribute per domain), and
5. the default sort order (distance or academic rating)

The experiment is conducted in one sitting. Participants are administered an online baseline survey and then randomized into one of the 72 treatment arms and given an endline survey that includes tasks to complete, such as ranking the schools and answering factual information about the schools described in the profiles. Participants cannot go back and change their responses to the baseline, but while they are completing the endline tasks they may toggle between the survey instrument and the school profile information display. The study will record response times as well as responses to survey items themselves.

The study will allow the researchers to estimate the impact of each of these factors on the way that parents actually rank schools (consistency with stated preferences, and whether the factors push parents toward favoring one domain over another), as well as their ability to comprehend the information and their overall attitudes toward the information (such as whether they found it useful).

The information will be used to inform a guide for school districts and other entities seeking to provide choice information to parents via online tools.

Intervention

When consumers shop for schools they seek out several sources of information. One important source is an official website with schools profiled. The interventions under study are variations on these official online information displays. The basic display will be a map centered on the user's hypothetical home address, showing 16 nearby schools as icons. Below the map is a list of the same 16 schools with a one-line profile for each, showing basic indicators. In some variations (see below), there is additional information on each school.

The treatment factors that vary are:
1. the format (numbers, numbers + graphs, or numbers + icons),
2. the source of information (objective indicators or objective + subjective indicators),
3. the presence or absence of a reference point, namely the district-level mean value for each indicator
4. the number of attributes per domain and disclosure method (one attribute per information domain, multiple attributes per information domain, or multiple attributes with progressive disclosure via user-initiated click-through to see beyond the first attribute per domain), and
5. the default sort order (distance or academic rating)

Experimental design

The design a factorial experiment that we will analyze using a hierarchical Bayesian model, estimated with Stan software.

Design details

Randomization method
Randomization is being automated as part of the system that delivers the online survey.
Randomization unit
The study randomizes individual respondents.
Clusters
Not applicable.
Observations
3,240 respondents
Arms
45 respondents in each treatment arm
Primary outcomes
There are three primary outcomes, each of which can be measured with multiple indicators: (1) understandability, or how comprehensible parents found the information presented; (2) usability, or how confident parents felt about using the information and how easy it was to use; and (3) effects on choices, including the weight placed on various school attributes and the extent to which ranking decisions align with parents' baseline preferences.

Papers from this trial

  1. Glazerman, Steven, Ira Nichols-Barrer, Jon Valant, Jesse Chandler, and Alyson Burnett. "Nudging Parents to Choose Better Schools: The Importance of School Choice Architecture" Washington, DC: Mathematica Policy Research, Working Paper No. 65, November 2018. https://mathematica-mpr.com/our-publications-and-findings/publications/nudging-parents-to-choose-better-schools-the-importance-of-school-choice-architecture?MPRSource=TCSide
  2. Steven Glazerman, Ira Nichols-Barrer, Jon Valant, Jesse Chandler & Alyson Burnett (2020) The Choice Architecture of School Choice Websites, Journal of Research on Educational Effectiveness, 13:2, 322-350, DOI: 10.1080/19345747.2020.1716905 https://doi.org/10.1080/19345747.2020.1716905
  3. Kassler, D., Nichols-Barrer, I., & Finucane, M. (2020). Beyond “Treatment Versus Control”: How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research. Evaluation Review, 44(4), 238-261. https://doi.org/10.1177/0193841X18818903 https://doi.org/10.1177/0193841X18818903

Cited by

No citation data available for this registration yet.

Keywords