Information, Accountability and Family-School Engagement: Evidence from Colombia
How to cite
Barrera-Osorio, Felipe; Deming, David. 2017. "Information, Accountability and Family-School Engagement: Evidence from Colombia." AEA RCT Registry. AEARCTR-0002192. https://doi.org/10.1257/rct.2192-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
The city of Manizales, capital of the Department of Caldas in central Colombia, is a mid-size Colombian city, with population close to half a million, and approximately 55,000 students in public basic education grades 1 to 11. Manizales is undertaking a major revision of its educational programs under the leadership of the mayor and in conjunction with a very active civic society. Authorities are interested in tackling the perceived low quality of education, as shown in national and international standardized tests. Our intervention aims to address the quality problem directly, and encompasses two experiments, one at the household level (information intervention) and one at the school level (family/school engagement intervention). The information intervention is designed to change the internal investment of the family as well as their relationship with the school. The family/school engagement intervention aims to increase engagement, trust, and communication between families and teachers/schools. We are interested to estimate average treatment effects and heterogeneity effects by test performance at baseline. The hypothesis is that parents from low-baseline performers have less information and may react more to treatment. We are interested in the independent impact of family treatment and school treatment, but also in the interaction between them. The unit of observation of the household experiment is household level; the unit of observation of the school experiment, will be the teacher level.
Intervention
Our study combines efforts to provide information to parents with a family engagement intervention, using an experimental design. The experimental design is divided in three phases. In the first and second phase, we provide report cards on school and student performance to parents. In the third phase, we will incorporate the family engagement component.
In the first phase, we randomly assigned families into three groups. In the first group (Treatment Group 1) we provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. The information is essentially a percentile rank (e.g. 50th percentile), presented in a way that is highly salient to all families. In the second group (Treatment Group 2) we provided families with a one-page report card that shows the average score of their child’s school in comparison to the average score across all schools in Manizales. Like treatment #1, the information is a percentile rank that is translated into a reader-friendly format. In the third group (Control Group) no information was provided.
We modified the information treatments from Phase 1 to Phase 2. In the first group (Treatment Group 1) we provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. Also included with the report card was a list of suggestions for parents to engage with their children’s education. In the second group (Treatment Group 2) we provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade in the entire city. The same list of suggestions provided to parents in Treatment Group 1 was provided to parents in Treatment Group 2. In the third group (Control Group) no information was provided.
In Phase 3 of the study, we add a family engagement component designed to improve trust and communication between the family and the school. Teachers were randomly assigned to two group. In the Treatment Group, teachers received the test score information collected for the students in their class. Teachers also received a list of suggestions to promote improve family-school engagement, with two components. First, teachers received suggestions for teachers on how to improve communication with families. Second, teachers received suggestions on how to encourage families to engage with their children’s education outside of the school. This is the same list of suggestions that is provided to parents in the treatment groups in Phase 2. In the Control Teachers, teachers did not receive test score information or the list of suggestions to promote family-school engagement.
Experimental design
Phase 1: Pilot Study (October 2014 – January 2015)
In April 2014, the project collected language and math test scores of students in grades 4 and 5. The first phase of the experiment consisted of two information treatments plus a control group. These students were randomly assigned into three groups: two treatment groups and one control group.
• Treatment Group 1: Individual student performance – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. The information is essentially a percentile rank (e.g. 50th percentile), presented in a way that is highly salient to all families.
• Treatment Group 2: School average performance – We provided families with a one-page report card that shows the average score of their child’s school in comparison to the average score across all schools in Manizales.
• Control Group: No information was provided.
To conduct the pilot, we randomly selected within the three groups a subsample to conduct the home visits.
Phase 2: Expanded Study (February 2015 – April 2016)
Phase 2, the expanded study, was conducted between February 2015 and April 2016. In June 2015, the project collected language and math test scores of the students who were randomly assigned to the treatment or control conditions in Phase 1. In addition, the project collected language and math test scores of students who were in grade 4 in the schools targeted in Phase 1. These new students were randomly assigned into three groups: two treatments and a control group. We modified the information treatments from Phase 1 to Phase 2:
• Treatment Group 1: Individual student performance in the school – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade and school. Also included with the report card was a list of suggestions for parents to engage with their children’s education.
• Treatment Group 2: Individual student performance in the city – We provided families with a one-page report card that shows their child’s performance, as well as their relative position to the average performance of students in the same grade in the entire city. The same list of suggestions provided to parents in Treatment Group 1 was provided to parents in Treatment Group 2.
• Control Group: No information was provided.
We conducted a second home visit in November 2015. During this home visit, we provided information to parents of the students who had been assigned to either treatment group in Phase 1. The information provided was based on the modified treatment arms described above. We also provided information to the parents of the students who had been assigned to either treatment group in Phase 2. These home visits followed the same format as the home visits in Phase 1.
Phase 3: Full Study (May 2016 – August 2017)
The third phase of the study is different to the pilot study in two main senses. First, we expand the sample to all students in the grades of interest. As we described, in 2014 and 2015 students from 31 schools in grades 4, 5, and 6 were included. Beginning in 2016, we will not only follow those students that are in grades 5 and 6, but we will also include all students in grades 4, 5 and 6 that are enrolled in one of the 31 schools. We will not be following students who were in grade 5 in Phase 1 and are in grade 7 in Phase 3. Second, we will add a family engagement component.
In May 2016 we administered language and math tests to all students in grades 4, 5 and 6 within the 31 schools. This sample includes all students who were assigned to one of the three information intervention groups in Phase 1 and 2. Students who did not participate in Phase 1 or Phase 2 were not assigned to an information intervention group.
In addition to continuing to collect follow-up information for the information intervention, we included a family engagement component designed to improve trust and communication between the family and the school. In mid 2016, teachers were randomly assigned to one of two groups:
• Treatment Group: Family Engagement Intervention – Teachers received the test score information collected in May 2016 for the students in their class. Teachers also received a list of suggestions to promote improve family-school engagement, with two components. First, teachers received suggestions for teachers on how to improve communication with families. Second, teachers received suggestions on how to encourage families to engage with their children’s education outside of the school. This is the same list of suggestions that is provided to parents in the treatment groups in Phase 2.
• Control Group: No Family Engagement Intervention – Teachers will not receive test score information or the list of suggestions to promote family-school engagement.
At the end of each academic term we administered language and math tests to all students in the sample.
Design details
Cited by
No citation data available for this registration yet.