Evaluation of the District of Columbia Public Schools Family Engagement Partnership

How to cite

McKie, Allison; Glazerman, Steven. 2016. "Evaluation of the District of Columbia Public Schools Family Engagement Partnership." AEA RCT Registry. AEARCTR-0001370. https://doi.org/10.1257/rct.1370-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Allison McKie* , Steven Glazerman

* Primary investigator

Registry ID
AEARCTR-0001370
DOI
https://doi.org/10.1257/rct.1370-2.0
First registered
Last updated
Status
Withdrawn
Trial dates
2014-01-15 – 2015-12-18
Countries
United States of America (District of Columbia Public Schools (Washington, DC))
JEL codes
I21 Analysis of education
Registry record
http://www.socialscienceregistry.org/trials/1370

Abstract

The District of Columbia Public Schools (DCPS) partnered with the Flamboyan Foundation to implement the Family Engagement Partnership (FEP), a two-year school-based intervention that aims to increase family engagement and, ultimately, student achievement. As part of this effort, DCPS selected Mathematica Policy Research, a nonpartisan research firm, to serve as the independent evaluator of FEP. Mathematica conducted a randomized controlled trial with a multi-step screening process to estimate the impacts of FEP on family engagement, student achievement, and other key outcomes. The treatment in this intent-to-treat evaluation was the invitation to apply to FEP, not necessarily the implementation of FEP. In addition to using administrative data to measure student and teacher outcomes, we planned to develop and administer surveys to assess family engagement activities and outcomes. Teacher surveys conducted in spring 2015, spring 2016, and spring 2018 would allow us to examine whether FEP activities differ from usual practices in control schools and estimate impacts on teacher satisfaction and perceptions of student-teacher relationships. We planned to conduct a parent survey in summer 2016 and summer 2018 to measure family engagement outcomes from the parents’ perspective. Findings were to be presented in a 2019 report. However, the study sponsors decided to end the evaluation early due to statistical power considerations. Preliminary findings from this evaluation and another study’s recent findings suggested that impacts of the intervention may be difficult to detect in part because they may be modest in size and not statistically significant with the sample of schools participating in the study.

Intervention

FEP is a two-year, school-based intervention that aims to increase family engagement and, ultimately, student achievement, via activities such as home visits, academic parent-teacher teams, and ongoing teacher communication with families.

Experimental design

Mathematica designed a randomized controlled trial with a multi-step screening process. In Step 1, DCPS screened schools based on extant information for suitability in FEP. This step did not involve outreach to candidate schools themselves. Step 2 involved minimal outreach to schools of marginal suitability to gauge interest and capacity. The outreach was not rigorous enough to raise schools’ hopes or change their behavior. Schools passing this screen, however, had to consent to participate in the study. At the conclusion of this step, DCPS provided Mathematica with a list of schools to randomly assign.

In Step 3, the Mathematica evaluation team randomly assigned half of the schools to a treatment group and half to a control group. Mathematica first used school-level data to create pairs of schools that were similar on observable characteristics, such as average student achievement and student demographics. We then consulted with DCPS to ensure that district staff and program implementers who were familiar with the schools deemed each proposed pair of schools to be similar. Once the matched pairs were agreed upon, we randomly assigned one school in each pair to be invited to apply for FEP (treatment group); the other school was not invited to apply (control group).

FEP staff then rigorously screened the treatment schools and worked with the treatment schools that were selected for the intervention. The evaluation team planned to follow all study schools over time. In this intent-to-treat evaluation, schools assigned to treatment remain in the treatment group for the study, regardless of whether they implemented FEP.

Design details

Randomization method
Mathematica conducted randomization within matched pairs of schools using a random number generator.
Randomization unit
School.
Clusters
40 schools.
Observations
Approximately 10,200 students with non-missing administrative data, 400 teachers in the teacher survey sample, and 400 parents in the parent survey sample.
Arms
As planned: 20 treatment schools and 20 control schools. As implemented: 9 treatment schools and 9 control schools.
Primary outcomes
The main outcomes are students’ attachment to school, student achievement, teacher retention, teacher performance, and family engagement.

Cited by

No citation data available for this registration yet.

Keywords