On Rating Scales in Subjective Performance Evaluations – The Effect of a Dummy Category II
How to cite
Vogt, Thomas; Stangl, Tobias; Sliwka, Dirk; Thonemann, Ulrich W.. 2018. "On Rating Scales in Subjective Performance Evaluations – The Effect of a Dummy Category II." AEA RCT Registry. AEARCTR-0003029. https://doi.org/10.1257/rct.3029-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
A natural field experiment is conducted to investigate the influence of a dummy evaluation category at the bottom of a feedback scale on effort provision and performance. Subjects work on a real effort task in two successive periods. A performance dependent bonus is paid for both periods. Performance is evaluated using three evaluation categories. The first category is awarded to the highest performing subjects while the lowest performing subjects are evaluated with a three.
Subjects are randomly assigned to one of three treatments. In treatment ND no dummy evaluation category is shown, i.e. subjects are shown the actual three evaluation categories. In treatment CD a 4th dummy evaluation category is shown. In treatment TD a 4th dummy evaluation category is shown and the non-usage of the 4th evaluation category is made transparent.
We hypothesize that average effort provision and performance is higher in treatment CD compared to treatment ND in the second period. We also hypothesize that average effort provision and performance is higher in treatment CD compared to treatment TD in the second period. Moreover, we hypothesize that average effort provision and performance is higher in treatment TD compared to treatment ND in the second period. We expect these effects to be strongest for those ranking lowest (third).
Experimental design
The first period is the same across treatments – subjects are asked to work on a real effort task. Subjects are informed that they receive a performance based bonus payment but no information on the evaluation scale is given.
In the second period, subjects learn about the evaluation scale and receive private feedback on their own evaluation of the performance in the first period before they can work again. Actual evaluations are based on the relative performance of subjects and follow exactly the same procedure in all three treatments such that only categories 1-3 are actually awarded. Subjects are not informed about the specific details of the evaluation procedure but they learn that about 30% of subjects are awarded the best grade and 40% the second best grade. In treatment ND and TD they are informed that 30% of the evaluations are in category 3. In treatment CD they learn that 30% of the evaluations are either in category 3 or category 4.
Subjects are informed that the evaluation scale of the first period is used for the bonus payments in the second period.
We are varying whether a 4th dummy evaluation category is shown in the evaluation scale in the second period (treatment ND vs. CD/TD). Additionally, we are varying whether the non-usage of the 4th dummy evaluation category is made transparent (treatment CD vs. TD).
Design details
Cited by
No citation data available for this registration yet.