The effects of incentives in non-routine analytical team tasks
How to cite
Schudy, Simeon; Schindler, David; Englmaier, Florian; Grimm, Stefan. 2021. "The effects of incentives in non-routine analytical team tasks." AEA RCT Registry. AEARCTR-0008073. https://doi.org/10.1257/rct.8073-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Despite the prevalence of non-routine analytical team tasks in modern economies, little is known about how incentives influence performance in these tasks. In an earlier field experiment with more than 3,000 participants, we documented a positive effect of bonus incentives on the probability of completion of such a non-routine team task (a real-life escape game). To investigate which dimensions of effort are most affected by team incentives, this additional study implements an experiment that closely resembles real-life escape games and contains all relevant aspects of them. In particular, in this study, teams again have to solve a series of complex and novel problems, need to collect and recombine information, and have to solve analytical and cognitively demanding quests that require thinking outside the box. The experiment allows us to determine how incentives impact performance by exogenously varying the most important margins of team production that incentives can affect.
The experiment is based on a board game version of a real-life escape game. Following a natural field experiment (involving a real-life escape game) we conducted earlier, we implement three treatments: In BGControl, teams are not provided with any monetary incentives and have 60 minutes to complete the task for a flat payment (irrespective of success). In BGGain45, subjects can receive an additional team bonus if they succeed in completing the task in less than 45 minutes, and treatment BGLoss45 implements the same treatment but with a loss frame. The additional experiment has three main objectives. First, we test if bonus incentives causally affect whether teams assign the most skilled team member to a specific sub-task (skill-to-task matching) and analyze how this subsequently maps into performance. Second, we investigate the causal effect of incentives on the likelihood of team members sharing relevant information (information sharing) to facilitate task completion. Third, we study the causal effect of incentives on communication and provide additional causal evidence on how communication subsequently maps into performance.
Intervention
We implement three across-subject treatments: a control condition, an incentive condition in which a bonus incentive is framed as a gain, and an incentive condition in which a bonus incentive is framed as a loss.
Additionally, we perform three within-subject treatments (described in the experimental details).
Experimental design
Our experiment is based on a board game version of a real-life escape game. The setting resembles the same important features as our field study while it is also implementable in the laboratory. We alter the board game to explicitly study the causal effects of team incentives on the three effort dimensions respondents of a survey conducted earlier rated most important:
First, we test if incentives causally affect whether teams assign the most skilled team member to a specific sub-task (skill-to-task matching) and analyze how this subsequently maps into performance.
Second, we investigate the causal effect of incentives on the likelihood of team members sharing relevant information (information sharing) to facilitate task completion. Additionally, we show how more or less information sharing subsequently translates into completion times.
Third, we study the causal effect of incentives on communication and provide additional causal evidence on how communication subsequently maps into performance.
Following our previously conducted natural field experiment, we also implement three treatments across subjects. In BGControl, teams are not provided with any monetary incentives and have 60 minutes to complete the task for a flat payment of 15euro (irrespective of success). In BGGain45, subjects can receive an additional team bonus of 30 euro if they succeed in completing the task in less than 45 minutes, and treatment BGLoss45 implements the same treatment but with a loss frame. Teams always consist of three participants.
The experiment consists of three parts (part I, part II, and part III), and each part contains several sub-parts (e.g. part I.2 denotes subpart 2 of part I). Before the start of the experiment, participants undergo Raven's progressive matrices test. Upon completion of the test, participants receive feedback only about their own test score. Afterward, they receive several materials, and each team member additionally receives a text that contains (different) information which can be shared with other team members in part II.2.
The information to share is part of a longer text with otherwise entertaining yet not helpful information and participants will not be able to deduct that some information may be valuable until they reach part II.2.
Subjects then commence with part I.1. The exact design and tasks of all parts are described in the additional (hidden) section Experimental Design. After completion of this part, subjects advance to part I.2 and subsequently to part II.1, which we use to study the effects of incentives through verbal communication. We deliberately design parts I.2 and II.1 to be similar, yet challenging to subjects, and prohibit subjects to use verbal communication in one of the two parts. This is announced only at the beginning of the respective part and enforced by the threat of exclusion through the experimenter. We then compare performance in part I.2 relative to performance in part II.1 between our incentive treatments and control. The (exogenously created) absence of verbal communication thus allows us to determine the causal effect of communication on performance, and whether this effect is dependent on the provision of monetary incentives in a difference in differences analysis. To avoid order effects, we randomly vary whether verbal communication is prohibited in part I.2 or in part II.1, and additionally flip the two parts for half of all sessions.
In part II.2, subjects can share the information distributed before the start of the experiment. Importantly, the information provided is sufficient, but not necessary to arrive at the correct solution. Alternatively, subjects can decide to not share their information and use the materials provided to work on the part's solution. By comparing the differences in how much information is shared across treatments with and without incentives, we can determine the causal effect of incentives on information sharing. Additionally, we will be able to see to what extent information sharing correlates with eventual performance in task II.2.
In part III, subjects are asked to select one team member for a task that requires logical reasoning. Participants may disclose their score from the Raven test taken prior to the experiment to aid the team's choice. They are, however, not explicitly reminded of the test and do not receive suggestions that this test may have predictive power for performance in the task to follow. While the chosen team member works on the logical reasoning task, secluded from the rest of her group, the other team members work on other tasks needed to finish the experiment. By comparing the differences in cognitive test scores of the team member chosen and the team member with the highest score between our incentive conditions, we can estimate the causal effect of incentives on skills-to-task matching. Additionally, we can correlate the aforementioned test differences with subsequent performance in the logical reasoning task. After the conclusion of part III, subjects fill in some questionnaires and are paid and subsequently dismissed from the laboratory.
To avoid time trends in the data affecting our results, we plan to run three sessions concurrently, such that each treatment will be conducted at the same time and day.
We plan to recruit 360 participants to form 120 teams consisting of three members each and run the experiment in Munich (Germany), Tilburg (The Netherlands), or elsewhere (depending on where Covid-19 restrictions allow us to recruit participants and run the experiment). Akin to the earlier framed field experiment, we assign two-thirds of teams to the incentive treatment (40 to BGGain45, 40 to BGLoss45), and one-third to BGControl. With a statistical power of 80 percent, this enables us to detect relevant effect sizes for the most important treatment comparisons: the effects of incentives (pooling BGGain45 and BGLoss45) on communication, information sharing, and skill-to-task matching. For instance, assuming similar performance distributions as in our framed field experiment, we are able to detect reductions in completion times of 3 minutes or larger.
Design details
Cited by
No citation data available for this registration yet.