Nudging Student Effort to Accurately Measure Student Ability: Maximizing the Value of Assessment Data through Nimble Interventions
How to cite
Baron, Juan; Adelman, Melissa. 2018. "Nudging Student Effort to Accurately Measure Student Ability: Maximizing the Value of Assessment Data through Nimble Interventions." AEA RCT Registry. AEARCTR-0003415. https://doi.org/10.1257/rct.3415-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Low-stakes, diagnostic student learning assessments are an important tool for countries in their response to the global “learning crisis.” Performance on such assessments reflects a combination of student learning and effort exerted. Therefore, test scores from such assessments may overestimate the true magnitude of learning gaps when differences in effort exist, and such effort differences may affect how assessment results should be interpreted (for example, the meaning of test score differentials across socioeconomic groups). Several randomized evaluations, primarily in the United States, have demonstrated that external incentives (monetary and non-monetary) and framing can significantly affect students’ performance on otherwise low-stakes tests (Braun, Kirsch, and Yamamoto 2011; Levitt et al. 2016; Gneezy et al. 2017). As countries advance in their reform of student assessment systems and continue to replace high-stakes assessments with low-stakes ones, the issue of whether low-stake assessments are capturing the reality of students’ knowledge and competences becomes more and more relevant, and even more so, if some groups of students (i.e. those from disadvantage backgrounds) tend to exert more or less effort than others.
The Dominican Republic (DR), which came in last both in PISA 2015 and in the TERCE regional assessment in 2013, has embarked on an ambitious assessment agenda that includes diagnostic learning assessments of 3rd, 6th, and 9th grades and their potential use for targeted in-service teacher training and other school-support activities. The results of these assessments will therefore inform the allocation of scarce resources, and DR Ministry of Education (MINERD) policymakers would like to maximize the extent to which they accurately measure learning. Relatedly, MINERD policymakers are interested in identifying low-cost, scalable approaches to increasing student effort in day-to-day, low-stakes tasks, where effort exerted can have a cumulatively important effect on how much students learn. This is particularly critical for poor and disadvantaged students, who are less likely to have direct experiences or role models of high effort leading to high achievement, and are also less likely to have parents who provide incentives for school achievement (Gottfried, Fleming, and Gottfried 1998; Austen-Smith and Fryer 2005).
Experimental design
The sample of schools for the study will be drawn randomly from the universe of public secondary schools in the Dominican Republic, stratified by district poverty rate. These school will be randomly assigned to 1 of 3 treatment groups, as such:
Treatment 1: Random sample of public secondary schools – will receive a visit from local MINERD staff to announce and carry out the awarding of external incentive (same day as assessment). Visit will be pre-announced to school director and teachers, without specifics on the purpose.
Treatment 2: Random sample of public secondary schools – will receive a packet addressed to school director and 9th grade teacher(s) with Growth Mindset exercise materials, instructions to conduct the exercise a few months prior to the assessment, and instructions requesting that a photo be texted or emailed to the Ministry showing the exercise in progress.
Treatment 3: Random sample of public secondary schools – will receive a packet addressed to school director and 9th grade teacher(s) with Self-Affirmation exercise materials, instructions to conduct the exercise a few months prior to the assessment, and instructions requesting that a photo be texted or emailed to the Ministry showing the exercise in progress.
A fourth group drawn in the same way will constitute the control group.
Design details
Cited by
No citation data available for this registration yet.