Management and Student Achievement: Evidence from a Randomized Field Experiment
How to cite
Fryer, Roland. 2017. "Management and Student Achievement: Evidence from a Randomized Field Experiment." AEA RCT Registry. AEARCTR-0002377. https://doi.org/10.1257/rct.2377-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study examines the impact on student achievement of implementing management training for principals in traditional public schools in Houston, Texas, using a school-level randomized field experiment. Across two years, principals were provided 300 hours of training on lesson planning, data-driven instruction, and teacher observation and coaching. The findings show that offering management training to principals significantly increases student achievement in all subjects in year one and has an insignificant effect in year two. We argue that the results in year two are driven by principal turnover, coupled with the cumulative nature of the training. Schools with principals who are predicted to remain in their positions for both years of the experiment demonstrate large treatment effects in both years - particularly those with principals who are also predicted to implement the training with high fidelity - while those with principals that are predicted to leave have statistically insignificant effects in each year of treatment.
Intervention
Year 1: Two (nonconsecutive) weeks of training (summer 2014), ongoing coaching and professional development from Chief Management Officer, set of high-quality interim assessments. 170 total hours of training (81% on content, 19% on systems).
Year 2: One-week training (summer 2015), ongoing coaching and professional development from Chief Management Officer, data monitoring systems provided by HISD to all schools, set of high-quality interim assessments. 130 total hours of training (38% on content, 62% on systems)
Principals and other applicable school leaders were expected to collect teacher lesson plans and provide feedback before plans are implemented, observe teachers in the classroom and give feedback at least biweekly, and work with teachers to analyze student data and create action plans.
Experimental design
The intervention relied on three levers of school management for the treatment to impact student test scores, the primary outcome of interest. The first lever was instructional planning. Teachers in treatment schools were to turn in lesson plans to principals, or another designated school official, before teaching the lesson in order to receive feedback in one-on-one coaching sessions. Principals and school leaders received training to identify well-made and poorly-made lesson plans and to effectively communicate ways to improve lesson plans with teachers. The second lever was data-driven instruction. Students were assessed every 6-8 weeks on interim assessments developed for this intervention (and usable for up to five years). Teachers were to review student results on the school district's data analysis platform, then draft an action plan. Principals were asked to meet with teachers to review and revise the action plans and help develop re-teaching strategies for struggling students as well as teaching strategies for differentiated instruction. Compliance was tracked via submission of action plans. The third lever was observation and feedback. Principals were asked to observe teachers every other week for 15-20 minutes and later have a meeting to discuss takeaways. In addition, principals were asked to identify at least one action-step for the teacher to take to improve instruction. Compliance was monitored by tracking principal-teacher observations.
The researchers estimated both intent-to-treat (ITT) effects and local average treatment effects (LATEs) on principals' training attendance and training implementation and students' test scores. The researchers also examined heterogeneous effects of treatment.
Randomization occurred at the school-level. The sample frame was a list of 174 eligible schools (132 elementary, 22 middle, 19 high school) provided to the researchers by HISD. Fifty-eight schools from this list (20 elementary, 20 middle, 18 high school) made up the sample population. The researchers used matched-pair randomization to assign schools to treatment and control groups. First, sample schools were ranked by the sum of their mean scores on the high stakes state exams in math and reading for the previous two years. Next, every two schools going down the ranking were grouped, and one was randomly assigned to receive the treatment while the other served as the control.
Design details
Papers from this trial
- Fryer, Roland G., Management and Student Achievement: Evidence from a Randomized Field Experiment (May 2017). NBER Working Paper No. w23437. https://ssrn.com/abstract=2971836
- Fryer, Roland G., Management and Student Achievement: Evidence from a Randomized Field Experiment (May 2017). NBER Working Paper No. w23437. https://ssrn.com/abstract=2971836
Cited by
No citation data available for this registration yet.