Incentives and Answer Quality for Open-Ended Questions

How to cite

Opitz, Saskia; Sliwka, Dirk; Vogelsang, Timo; Zimmermann, Tom. 2023. "Incentives and Answer Quality for Open-Ended Questions." AEA RCT Registry. AEARCTR-0010752. https://doi.org/10.1257/rct.10752-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Saskia Opitz* , Dirk Sliwka , Timo Vogelsang , Tom Zimmermann

* Primary investigator

Registry ID
AEARCTR-0010752
DOI
https://doi.org/10.1257/rct.10752-1.0
First registered
Last updated
Status
On going
Trial dates
2023-01-10 – 2023-12-31
Countries
Private
JEL codes
C83
Registry record
http://www.socialscienceregistry.org/trials/10752

Abstract

We investigate the question to what extent incentives have an impact on the quality of answers to open-ended survey questions. To this end, we conduct an online survey study on Prolific asking participants open-ended questions as well as using items from established scales. Moreover, we investigate whether giving participants an additional incentive to provide high-quality answers, i.e., informing them that we will hire some of the participants again for a well-paid task depending on the quality of their answers, can increase the answer quality for open-ended questions. To do so, we randomly provide half of the participants with this additional incentive.

Intervention

Depending on the quality of their answers, some participants in the treatment group are hired again for a well-paid task.

Experimental design

Participants have to answer three open-ended questions connected to performance feedback (e.g., “Imagine the following: You receive negative feedback from your boss knowing that you did not perform at your best. How do you think you would react?”) or unrelated to performance feedback (e.g., “Describe what you did last Sunday.“) We randomly allocate half of the participants into a treatment group, which receives an additional incentive they are informed about before answering the open-ended questions: depending on the quality of the answers (clear description and answer fit) to the open-ended questions, some participants are hired again for a well-paid task. The control group does not receive this incentive.

Two human evaluators read the submitted answers to the open-ended questions and rate their quality based on clarity of the description as well as answer fit to the question. The two best performing participants in this respect will be hired to perform a real-effort task, which will be paid about 150% of the usual hourly wage paid on Prolific.

After the open-ended questions, we further elicit conscientiousness, social comparison and feedback orientation using established scales. Additionally, we elicit education level, age and gender.

Design details

Randomization method
We will randomize participants into treatment or control group using a random number generator such that there is an equal number of subjects in each group.
Randomization unit
Individual subject
Clusters
The number of clusters is the same as the number of observations (please see below).
Observations
300 individual subjects
Arms
150 individual subjects control, 150 individual subjects treatment
Primary outcomes
The primary outcome variable is the quality of the given answers, i.e., clarity and answer fit as rated by two human evaluators.

Cited by

No citation data available for this registration yet.

Keywords