Subjective versus Objective Performance Pay for Teachers
How to cite
Brown, Christina; Andrabi, Tahir. 2019. "Subjective versus Objective Performance Pay for Teachers." AEA RCT Registry. AEARCTR-0003835. https://doi.org/10.1257/rct.3835-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study looks at the effects of providing teachers raises based on their students' test scores versus their principal's rating of them as compared to a flat raise. We measure the effect on teacher behavior in and out of the classroom, teacher retention, and student performance. While subjective performance pay is easier to implement and can be more successful at addressing multi-tasking concerns, there is a worry that principals may be biased against certain groups or show favoritism toward teachers they are socially connected to. We look for evidence of gender discrimination and favoritism toward connected teachers as well as heterogeneity in treatment effects by more or less biased principals. Finally, we introduce an additional randomization as to how often principals observe certain teachers to vary the principals' accuracy of information. We look to see if principals accurately update and if teachers in response work harder when their effort is more observable.
Intervention
A. Incentive Treatments
Control: Flat raise - All teachers will receive the same raise (about 7%) at the end of the calendar year. This is equal to about twice the cost of living inflation.
Treatment 1a: Subjective Performance Raise - All teachers will be rated by their direct supervisor (usually a principal or vice principal) on the teacher's "effort in improving students' academic performance'' on a scale from 0-100. Supervisors will give a short justification for their rating. Within the school, teachers will then be ranked according to this subjective score, placed into 5 bins and given a raise from 0-15% based on their ranking.
Treatment 1b: Subject Performance Raise - All teachers will be rated by their direct supervisor (usually a principal or vice principal) on a number of criteria determined by HR, such as timeliness, communication and professional integrity on a scale from 0-100. Supervisors will give a short justification for their rating. Within the school, teachers will then be ranked according to this subjective score, placed into 5 bins and given a raise from 0-15% based on their ranking.
Treatment 2: Objective Performance Raise - All teachers receive a raise based on percentile value-added (Barlevy and Neal, 2016) -- their students' average percentile within their lagged percentile comparison group on the standardized exam at the end of each semester. Teachers will then be ranked within the school based on this measure of value added, placed into 5 bins and receive a raise varying from 0-15%.
On average the raise percentage is the same across all treatments.
B. Observation Treatment
Control: Status quo – Teachers observed at the normal frequency by their supervisor, typically once a semester
Treatment: Frequent observation – Principals are requested to observe teachers for at least 10 minutes once every three weeks
Experimental design
We randomly assign teachers to receive a variety of incentive contracts and cross randomize with how often their manager observes them. The key specification will look at the effect of each scheme on test scores and teacher effort. We will also look at the interaction with being observed more to determine whether that increases the perceived "bite" of the subjective incentives and therefore, teacher effort in that treatment.
We will take the data to a classical principal-agent model to understand how the perceived noisiness and distortion of each incentive system helps us understand the choices teachers make in terms of the extent and direction of effort change.
In March we will conduct a survey to understand what teachers believed about each incentive system. If teachers interpreted both treatments similarly, we will collapse those two treatments.
Note: The listed design details are being pre-specified after the treatment implementation is complete but before endline data was finished being collected. PIs had reviewed midline student grade outcomes and video data prior to pre-specification.
Design details
Cited by
No citation data available for this registration yet.