On Rating Scales in Subjective Performance Evaluations – The Effect of a Dummy Category

How to cite

Vogt, Thomas; Thonemann, Ulrich; Stangl, Tobias; Sliwka, Dirk. 2018. "On Rating Scales in Subjective Performance Evaluations – The Effect of a Dummy Category." AEA RCT Registry. AEARCTR-0002736. https://doi.org/10.1257/rct.2736-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Thomas Vogt* , Ulrich Thonemann , Tobias Stangl , Dirk Sliwka

* Primary investigator

Registry ID
AEARCTR-0002736
DOI
https://doi.org/10.1257/rct.2736-1.0
First registered
Last updated
Status
In development
Trial dates
2018-03-19 – 2018-06-30
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/2736

Abstract

A natural field experiment is conducted to investigate the influence of a dummy evaluation category at the bottom of a feedback scale on effort provision and performance. Subjects work on a real effort task in two successive periods. A performance dependent bonus is paid for the first period. Performance is evaluated using three evaluation categories. The first category is awarded to the highest performing subjects while the lowest performing subjects are evaluated with a three. A fixed wage is paid in the second period.

Subjects are randomly assigned to one of two treatments. In treatment ND no dummy evaluation category is shown, i.e. subjects are shown the actual three evaluation categories. In treatment CD a 4th dummy evaluation category is shown.

We hypothesize that average effort provision and performance is higher in treatment CD in the second period. We expect these effects to be strongest for those ranking lowest (third).

Experimental design

The first period is the same across treatments – subjects are asked to work on a real effort task. Subjects are informed that they receive a performance based bonus payment but no information on the evaluation scale is given.

In the second period, subjects learn about the evaluation scale and receive private feedback on their own evaluation of the performance in the first period before they can work again. Actual evaluations are based on the relative performance of subjects and follow exactly the same procedure in both treatments such that only categories 1-3 are actually awarded. Subjects are not informed about the specific details of the evaluation procedure but they learn that about 30% of subjects are awarded the best grade and 40% the second best grade. In Treatment ND they are informed that 30% of the evaluations are in category 3. In treatment CD they learn that 30% of the evaluations are either in category 3 or category 4.
We are varying whether the 4th dummy evaluation category is shown in the evaluation scale in the second period.

Design details

Randomization method
Stratification method
Randomization unit
individual
Clusters
1000
Observations
1000
Arms
Control (ND): 500 subjects CD: 500 subjects Note: There can be slight changes in the number of subjects in each treatment due to selective attrition in the second part. However, we document drop-outs and test whether these are systematic.
Primary outcomes
The number of cover sheets entered correctly on the individual level (individual performance), the number of cover sheets entered on the individual level (individual effort provision), questionnaire data (post)

Cited by

No citation data available for this registration yet.

Keywords