This project assesses the extent to which information about student performance on standardized tests influences perceptions of the recent state takeover of Houston Independent School District (HISD) by Texas Education Agency. Specifically, we test whether providing information to survey respondents about test performance in the HISD schools most directly impacted by the takeover—and growth in performance in these schools—influences ratings of the reform (overall and its individual components), ratings of the newly installed district leadership, and stated preferences of which future actions respondents would prefer policymakers pursue. The survey experiment also collects data on respondents' perceptions of the political process that led to the takeover and how it was implemented. We test whether the role of information in shaping respondent preferences is moderated by proximity to HISD as well as the mediating role of process perceptions.
Intervention
After consenting to participate, this survey begins with a series of screener questions and some baseline demographic questions.
At the beginning of the survey experiment, all respondents are provided with a short description of the 2023 takeover of HISD by Texas Education Agency. Half the respondents receive only this information (control group), and half the respondents are randomly assigned to receive additional information (treatment group) about student performance on standardized tests in different sets of schools.
After viewing this information, both groups answer a series of questions related to their perception of the state takeover (overall and its individual components), HISD district leadership, the process that led to the takeover, and a 2024 bond measure that would provide $4.4 billion to finance improvements within HISD schools.
The survey experiment concludes with both groups answering a set of demographic questions.
Experimental design
This is a simple two-armed randomized survey experiment. 50% of participants will be randomized to view one block of text describing the 2023 TEA takeover of HISD (control group), and 50% will be randomized to view this text as well as additional text and figures that include information about student performance on standardized tests within different groups of Texas schools (treatment group). Both groups will then answer the same set of questions about their perceptions of the 2023 TEA takeover of HISD.
Approximately 500 Texas residents will be recruited from an online survey platform to participate and will take the full survey. Approximately 1000 additional participants will be recruited from a national pool of respondents and will take an abridged survey (see both survey instruments in "Supporting Documents and Materials").
Design details
Randomization method
The randomization will be embedded into the survey using the “Randomizer” function in Qualtrics.
Randomization unit
Individual
Clusters
1500
Observations
1500
Arms
1500
Primary outcomes
The primary outcome of interest is an index summarizing respondents rating of the state takeover and its individual components.
Secondary outcomes
The secondary outcomes of the Texas study will include a series of indices that assess different dimensions of respondents' perceptions of the state takeover of HISD, including 1) An index summarizing respondents' overall rating of the state takeover of HISD; 2) respondents' overall rating of NES reforms related to teacher and principal compensation and evaluation; 3) respondents' overall rating of NES reforms related to classroom instruction; 4) respondents' overall rating of NES reforms related to school culture and staffing; 5) respondents' overall rating of NES reforms related to school operations.
Additionally, in both the Texas and National studies, we will convert respondents' stated preferences about when TEA should restore governing responsibilities to the elected HISD school board into a binary variable based on the median response of the control group, and report whether the informational intervention influenced respondents to prefer a longer or shorter takeover.
Similarly, in both the Texas and National studies we will convert respondents' stated preferences about the conditions under which TEA should restore governing responsibilities to the elected HISD school board into a binary variable based on the median response of the control group, and report whether the informational intervention influenced respondents to prefer more or less restrictive conditions for ending the takeover.
Additionally, in both the Texas and National studies, we will convert respondents' stated preferences about the $4.4 billion 2024 HISD bond measure into a binary variable based on the median response of the control group, and report whether the informational intervention influenced respondents to express more or less support for the bond measure.
Finally, we will conduct a moderation and mediation analyses. For our primary moderation analysis, we will compare the influence of information on respondent preferences based on distance from Houston. First, we will compare the impact of information on the preferences of survey respondents who take the survey in Texas versus those who live outside of Texas. If we have at least 150 survey participants who live in Harris County, Texas, we will compare the impact on Harris County residents versus respondents who take the survey in Texas versus those who live outside of Texas. This is a slight change from our original pre-registered moderation analysis plan (see trial history), when we anticipated being able to target the survey to Harris County residents only (we were able to restrict the sample by state, but not county).
We will examine participants ratings of the process that led to the takeover as a mediator of their ratings of the takeover.
Exploratory analyses will examine the impact of information in responses to particular questions or subquestions.
Cited by
No citation data available for this registration yet.