The Success for All Model of School Reform
How to cite
Quint, Janet; Zhu, Pei. 2016. "The Success for All Model of School Reform." AEA RCT Registry. AEARCTR-0001136. https://doi.org/10.1257/rct.1136-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Improving the reading performance of at-risk elementary students is one of the greatest challenges confronting American education. Success for All (SFA), a program aimed at ensuring that every child learns to read well in the elementary grades, is one of the best-known school reform models. SFA emphasizes phonics instruction for beginning readers and comprehension for all students. It incorporates cross-grade ability grouping for reading, highly structured lessons that make extensive use of cooperative leaning, frequent assessments of students’ progress, tutoring, staff teams to solve problems that go beyond academics, professional development for teachers, and a program facilitator at each school. On the basis of the program's strong record of success, the Success for All Foundation (SFAF) received a scale-up grant through the U.S. Department of Education’s Investing in Innovation (i3) initiative to expand the intervention to hundreds of additional schools.
MDRC conducted an independent evaluation of the scale-up initiative, using a cluster random assignment design in which 37 schools in five school districts that participated in the scale-up effort were assigned at random to a program group (19 schools), which put in place the intervention, or to a control group (18 schools).which implemented "business as usual" reading instruction. This design supports causal impact findings for the average school assigned to SFA.
MDRC collected and analyzed data from a variety of sources — teacher logs, principal and teacher surveys, and rating sheets completed by the coaches — to examine the implementation experiences of the schools participating in the study. The evaluation indicates that while SFA was implemented with adequate fidelity at the large majority of schools that adopted it, resource constraints prevented some schools from putting in place some key components, including the program's computerized tutoring for students needing assistance beyond the classroom. The evaluation also compared instruction and other characteristics of program and control schools. Program schools were unique in placing students in cross-grade ability groups for reading, and teachers in program group schools used cooperative learning as an instructional method more frequently than their counterparts in control schools. The two groups of schools did not differ greatly along other dimensions that were measured.
The impact evaluation focuses on a primary analysis sample of students who enrolled in kindergarten in the study schools and remained in these schools over three years, through second grade. The confirmatory analysis found that in Year 3, SFA produced a positive and statistically significant impact on one measure of phonics knowledge; the impact on a second phonics measure was positive, but not statistically significant. In Year 3, students in SFA schools did not outperform their control-school counterparts on measures of fluency or comprehension.
An exploratory analysis indicates that for a group of special concern to policymakers and practitioners -- students who started school with low preliteracy skills--the program had notable third-year effects. For those second-graders who, at the start of kindergarten, scored in the bottom half of the sample in terms of their knowledge of the alphabet and their ability to sound out words, the program produced positive and statistically significant gains on measures of phonics skills, word recognition, and reading fluency; the impact on comprehension was also positive, although not statistically significant. The program did not significantly affect these outcomes for the subgroup of students who started kindergarten in the top half of the sample in terms of phonetic skills.
SFA did not produce statistically significant effects on the average reading performance of students in grades 3-5. There were no consistent patterns or significant findings of program impacts on special education identification rates and retention rates.
A cost analysis made use of the random assignment design within one case study district to assess the extent to which the district's SFA schools required additional resources, relative to those used for reading program in the control group schools. These additional costs were relatively modest: Direct expenditures for the school-based reading facilitator's time, after-school tutoring time, materials, and professional development were estimated to cost $119 more per student per year in SFA schools than in control group schools. Adding to this the additional time that SFA principals devoted to the program, additional time that coaches and teachers spent on training, the extra cost of space for storing SFA materials, and other factors, program group schools in this district spent about $227 worth of total resources per student per year more than control group schools to implement their respective reading programs.
Through the fourth year of the i3 grant, SFA was put in place in 447 new schools and reached an estimated 276,000 students. These numbers fell below SFAF's ambitious goals but represent a notable achievement in a period of staff layoffs and other cutbacks in many schools and districts.
Intervention
First implemented in 1987, Success for All combines a challenging reading program, whole-school reform elements, and an emphasis on continuous improvement in order to improve the reading skills of children in the elementary grades. Specific features include a K-6 reading program that emphasizes phonics for beginning readers and comprehension for all students; briskly paced lesson plans and cooperative learning; cross-grade ability grouping for reading; computerized and individual tutoring; staff committees to address issues and promote parent and community involvement; and ongoing professional development.
Experimental design
MDRC employed a cluster random assignment research design in which 37 schools located in five districts were assigned at random: 19 to receive the program and the remaining 18 to constitute a “business-as-usual” control group.
The evaluation compared the reading outcomes of kindergartners in both sets of schools in the first year of the study, and of first-graders and second-graders in the next two years, using age-appropriate assessments that were administered individually to each child. The primary analysis sample consists of students who started kindergarten in the study schools in the 2011-2012 school year and remained in the schools through all three implementation years. In the program group, these are the students with the maximum possible exposure to the program. A secondary analysis was conducted of students who were tested in the spring of 2014 and who received varying amount of the SFA intervention, ranging from less than one year to three years. The impact results for this "spring " sample parallel those for the primary analysis sample and reflect the effects of SFA when taking student mobility into account. Finally, during the second and third years of the study, students in third through fifth grades were also assessed, this time in a group setting. Student scores on state standardized tests served as additional measures of reading skills for this "auxiliary sample" of somewhat older students. Students in the auxiliary sample were in grades 1 through 3 when the study began and therefore did not first learn to read "the SFA way."
Two types of analytic models are used for analyzing outcomes measured at different levels. For outcomes measured at the student level, such as reading test scores, the main impact estimation model is a two-level hierarchical linear model that accounts for the clustering of students within schools. For outcomes measured at the school level, such as schools' special education identification rate and retention rate, the estimation model is an ordinary least squares (OLS) regression model with the school as the unit of observation. Both types of models use data from all five study districts in a single analysis, treating districts as fixed effects in the model. Separate program impact estimates are obtained for each district and then are averaged across the five districts, with each district's estimate weighted in proportion to the number of SFA schools it has. Therefore, findings in this report represent the impact on student performance in the average SFA school within the five study districts. The results may not necessarily reflect what the program effect would be in the wider population of districts beyond the study sample.
.
Design details
Papers from this trial
- Quint, Janet C., Rekha Balu, Micah DeLaurentis, Shelley Rappaport, Thomas J. Smith, and Pei Zhu. 2013. The Success for All Model of School Reform: Early Findings from the Investing in Innovation (i3) Scale-Up. New York: MDRC. http://www.mdrc.org/publication/success-all-model-school-reform
- Quint, Janet C., Rekha Balu, Micah DeLaurentis, Shelley Rappaport, Thomas J. Smith, and Pei Zhu. 2014. The Success for All Model of School Reform: Interim Findings from the Investing in Innovation (i3) Scale-Up. New York: MDRC. http://www.mdrc.org/publication/interim-findings-investing-innovation-i3-scale
- Quint, Janet C., Pei Zhu, Rekha Balu, Shelley Rappaport, Micah DeLaurentis. 2015. Scaling Up the Success for All Model of School Reform: Final Report from the Investing in Innovation (i3) Evaluation. New York: MDRC. http://www.mdrc.org/publication/scaling-success-all-model-school-reform
Cited by
No citation data available for this registration yet.