Harnessing complementarities in the education production function

How to cite

Livingston, Jeffrey; Neckermann, Susanne; List, John. 2019. "Harnessing complementarities in the education production function." AEA RCT Registry. AEARCTR-0004577. https://doi.org/10.1257/rct.4577-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jeffrey Livingston* , Susanne Neckermann , John List

* Primary investigator

Registry ID
AEARCTR-0004577
DOI
https://doi.org/10.1257/rct.4577-1.0
First registered
Last updated
Status
Completed
Trial dates
2010-11-01 – 2015-03-27
Countries
United States of America (IL)
JEL codes
C93, D2, I21
Registry record
http://www.socialscienceregistry.org/trials/4577

Abstract

Recent research shows that complementarities between inputs in the education production function might be harnessed by implementing two policies simultaneously. However, budget constraints may make fully funding both policies infeasible, forcing school districts to choose between fully funding one policy or partially funding several. To address this issue, we conduct a field experiment where $90 of financial incentives are provided to either one input (students, parents, or tutors) or spread equally among multiple inputs to improve on a standardized test and meet other standards. The results suggest a limited amount of financial incentives has a greater impact on student achievement when directed at just one input.

Intervention

Tutors were hired by the school district with which we worked to help students prepare for the Illinois Standard Achievement Test (ISAT) in 2010-2011. We worked with these tutors to give incentives for students (and/or their parents and tutors) to meet a set of academic and behavioral standards. These include improving on a standardized test in the subject in which the student received tutoring (reading or math), keeping the course grade in the relevant subject at at least the level of the previous trimester (and above F), having no more than two unexcused absences, and having no all-day suspensions.

Experimental design

Our design consists of six treatment groups and one control group. The treatment groups allocate $90 of incentive payments across three inputs in various combinations – the student, the parents, and a tutor with whom the student worked on preparation for the 2011 ISAT. The first three treatment groups incentivize only one of the student, the student’s parent(s), or the tutor. If the student meets all of the required achievement and behavioral standards (described below), the input receives the entire $90. The fourth treatment group investigates whether this $90 is more effective if shared among the student and the parents. Each receives $45 if the standards are met. The fifth treatment group splits the incentive across all three parties; each earns $30 if the standards are met. The final treatment incentivizes a single input, students, but pays them only $30 if all standards were met.

There are four academic and behavioral standards students are required to meet to earn these incentives. They include: improving by at least one point (out of 20) on a standardized test that we created, improving the student’s grade in the relevant subject (reading or math) or maintaining it at its previous level and above a failing grade of F, having no more than two unexcused absences during the assessment period, and having no all-day suspensions during the assessment period.

Design details

Randomization method
Tutor-groups are randomly assigned to initial treatments blocking on tutor. We then improve the balance on school, homeroom teacher, subject (reading or math), grade level groups (K to 2nd, 3rd to 5th, and 6th to 8th), gender, race/ethnicity, number of meetings per week the group met with the tutor, and baseline test score using the following procedure. We randomly select a pair of tutor-groups to swap treatment assignments, calculate an overall imbalance score which is based on a hypothesis test that the randomization is balanced on each of the above variables, and keep this new assignment if it results in a lower imbalance score. This procedure is then repeated 500 times and we utilize the resulting assignment.
Randomization unit
Groups of students that met with the tutors. These groups range in size from one student to nine students. Most (87%) have two to six students.
Clusters
187 tutor groups, made up of one to nine students.
Observations
683 student-level outcomes.
Arms
Control: 151 students, 37 clusters Student only $90: 96 students, 28 clusters Parent only $90: 100 students, 28 clusters Tutor only $90: 101 students, 28 clusters Student and Parent $45 each: 89 students, 27 clusters All three $30 each: 98 students, 27 clusters Student only $30: 48 students, 12 clusters
Primary outcomes
1. Test score on a 20 question test (standardized by grade and subject, reading or maths) written by the experimenters On this test: 2. Percentage of easy questions answered correctly 3. Percentage of moderate questions answered correctly 4. Percentage of difficult questions answered correctly
Secondary outcomes
Test score on official standardized tests that the tutors were hired to help the students prepare for: 5. 2011 Illinois Standards Achievement Test (ISAT) 6. 2012 ISAT Dummy variables that equal one if a standard needed to earn incentives is met: 7. Have no more than two unexcused absences 8. Have no suspensions 9. Keep class grade in subject in which tutoring is received at its previous level or higher, and above F 10. A dummy variable that equals one if all four behavioral and achievement standards needed to earn rewards are met

Cited by

No citation data available for this registration yet.

Keywords