On Adolescent Human Capital Production: Evidence from a Structurally-Motivated Field Experiment Across Three School Districts

How to cite

Roy, Sutanuka; Price, Joseph; Cotton, Chris; Hickman, Brent; List, John. 2020. "On Adolescent Human Capital Production: Evidence from a Structurally-Motivated Field Experiment Across Three School Districts." AEA RCT Registry. AEARCTR-0006530. https://doi.org/10.1257/rct.6530-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sutanuka Roy* , Joseph Price , Chris Cotton , Brent Hickman , John List

* Primary investigator

Registry ID
AEARCTR-0006530
DOI
https://doi.org/10.1257/rct.6530-1.0
First registered
Last updated
Status
Completed
Trial dates
2014-05-01 – 2014-05-22
Countries
United States of America (Illinois)
JEL codes
C93, I25, 015
Registry record
http://www.socialscienceregistry.org/trials/6530

Abstract

We leverage a field experiment across three distinct school districts to identify key pieces of a structural model of adolescent human capital production. Our focus is inspired by the contemporary psychology of education literature, which expresses learning as a function of the ratio of the time spent on learning to the time needed to learn. By capturing two crucial student-level unobservables---which we denote as academic efficiency (turning inputs into outputs) and time preference (motivation)---our field experiment lends insights into the underpinnings of adolescent skill formation and provides a novel view of how to lessen racial and gender achievement gaps.

Intervention

Our research design builds on our study-choice and skill-formation framework to bring together experimental and structural methods to quantify unobserved student characteristics. Our strategy uses the student choice model as a basis for an econometric framework, where field experimental methods shape a data-generating process with the requisite sets of observables and variation to enable identification of the structural parameters at the individual level. This data-generating process is also carefully crafted to be as true to students' everyday academic choices and experiences as possible. With this in mind, we conducted the field experiment among fifth- and sixth-grade students in three Illinois school districts. We offered varying monetary incentives for completion of extra-curricular learning activities on a math study website that we developed.

Experimental design

We worked closely with fifth- and sixth-grade math teachers across the three participating school districts to implement the field experiment. The major research advantage to this partnership was that participation in the study was on an opt-out basis, allowing the research team to achieve a sample that was much more representative of the local populations our partner schools serve. The primary feature of the experiment was a website on which the students could complete up to 80 mathematics modules, referred to as quizzes, across five general topics. Students had access to the website for 10 days and could complete as many of the quizzes as they chose. Throughout the process, our web server monitored students' activities and tallied successful completion of quizzes. Piece-rate incentives were offered for task completion on the website, based on the number of quizzes completed successfully, rather than on time spent. We also measured proficiency using in-classroom mathematics assessments.

Our experimental design centers on randomized incentives. We adopted a linear piece-rate schedule with a constant marginal piece-rate that would be easy for adolescents to understand. We varied both the base payment, for showing up and completing the minimum amount of work, and the marginal piece-rate payment. No payments are offered until a child has passed 2 quizzes, which ensures a within-student panel for each individual. Each student was randomly assigned to one of three possible contract groups: (base pay: $15, bonus: $0.75), (base pay: $10, bonus: $1.00), and (base pay: $5, bonus: $1.25). Assignment was at the individual level, resulting in treatment variation within school, grades, and classrooms.

Our Experiment Timeline:

In summary, the experiment took place as follows.

1) Students took a pre-test and survey administered by their teachers in class.
2) Students were randomly assigned a wage contract, and provided with information about the experiment, including the website and their earnings potential.
3)For the next 10 days, student work on the website counted towards their compensation. Following the 10 day period, they were paid based on the number of quizzes successfully completed during that time.
4) Student took a post-test and a second survey administered by their teachers in class.

Design details

Randomization method
Assignment was at the individual level, resulting in treatment variation within school, grades, and classrooms. More specifically, our randomization algorithm first separated students into race-gender-school-grade bins. Within each bin it balanced on pre-test scores by ordering students according to their score and randomly assigning consecutive blocks of 3 similar-score students to contract groups 1, 2, and 3. The algorithm then repeated this process thousands of times, and selected the random assignment that minimized overall correlations between treatment status and balance variables. Our pre-exam materials were produced and organized in such a way that they could be collected from teachers and rapidly processed so as to allow for balancing on initial math proficiency during randomization. Exams were administered to students toward the end of the school week, and they were processed, randomization executed, and personalized instruction materials for each student were produced over the weekend for in-classroom delivery by math teachers the following Monday. Each student participant received a personalized letter in a sealed envelope, containing login credentials, instructions for accessing the website, and their individual piece-rate incentive contract. They were also promised prompt delivery of payments within 2 weeks following the end of the experiment.
Randomization unit
individual level
Clusters
1676 pupils.
Observations
1676 pupils.
Arms
contract with base pay 15, and bonus .75 has 557 pupils; contract with base 10 and bonus 1 has 559 pupils; contract with base 5 and bonus 1.25 has 560 pupils.
Primary outcomes
Number of Math Quizzes solved, Time Taken to Solve Quizzes and Estimating Structural parameters.

Cited by

No citation data available for this registration yet.

Keywords