On Rating Scales in Subjective Performance Evaluations - Performance Effects of Kindness

How to cite

Vogt, Thomas; Stangl, Tobias; Sliwka, Dirk; Thonemann, Ulrich. 2019. "On Rating Scales in Subjective Performance Evaluations - Performance Effects of Kindness." AEA RCT Registry. AEARCTR-0004385. https://doi.org/10.1257/rct.4385-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Thomas Vogt* , Tobias Stangl , Dirk Sliwka , Ulrich Thonemann

* Primary investigator

Registry ID
AEARCTR-0004385
DOI
https://doi.org/10.1257/rct.4385-1.0
First registered
Last updated
Status
In development
Trial dates
2019-10-21 – 2020-06-30
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/4385

Abstract

To incentivize employee performance, many companies employ performance appraisals that are tied to compensation. In such ratings, employees are typically evaluated on a specific predetermined scale. It is often observed that the lowest rating categories are unused resulting in compressed appraisals. So far, research on subjective evaluations often associated this behavior with rating bias. In this project however, we study the question whether compressed appraisals may be designed by companies intentionally adding unused lower categories in order to frame rating scales to avoid negatively reciprocal reactions of employees.

Experimental design

We conduct a laboratory experiment that involves the role of “employers” and “employees” resembling a standard work setting. In all treatments, subjects in the role of employees are randomly matched into groups with one subject in the role of employers. Employees work on a real effort task and receive a performance dependent bonus. Employers benefit from higher work effort as their payment (piece rate) depends on their employees’ performance. Employers determine the rating scale shown to employees when their performance is evaluated by the computer. The experiment consists of two stages. In the first stage, employers determine the rating scales shown to their groups of employees. Subsequently, employees learn the rating scale chosen by their employer and work on a real effort task afterwards.

We assess individuals’ ability in a pre-round, in which all subjects work on the same real-effort task. Subjects need to count the number “7” in blocks of randomly generated numbers (Berger et al. 2013). Based on this ability test, best performing subjects become employers. The remaining subjects become employees and are randomly assigned to an employer. Matching is anonymous and participants never receive information on the identity of other subjects. Using stratified sampling, we ensure equal ex ante performance across employee groups and equal number of employees per employer. To assure understanding of the experimental instructions, subjects must pass a comprehension quiz. Before the main task, subjects learn their role (employer or employee). The main task consists of two consecutive stages. In the first stage, employers determine the rating scales of the performance appraisal shown to their employees. Subsequently, employees learn the rating scale chosen by their employer. In the second stage, employees work on the real effort task of the pre-round for six rounds, each lasting 2.5 minutes. After the main task follows a questionnaire section.

Design details

Randomization method
Stratification method
Randomization unit
Individual
Observations
480 subjects total sample size
Arms
160 subjects in each treatment: 160 subjects NoDummy (control) treatment, 160 subjects Dummy treatment, 160 subjects DummyTransparent treatment Note: due to possible no-shows of subjects the final number of subjects per treatment might be less
Primary outcomes
Points for counting the number “7” in blocks of randomly generated numbers on the individual level (individual performance). Points for counting the number “7” in blocks of randomly generated numbers on the group level (group performance). The number of times the time-out button was used, i.e. leisure time was taken. Questionnaire data (post)

Cited by

No citation data available for this registration yet.

Keywords