A randomized experiment to test the effect of truth-telling incentivisation methods on survey responses
How to cite
Ceallaigh, Diarmaid O; Rohde, Kirsten I.M.; Kippersluis, Hans van.
2019. "A randomized experiment to test the effect of truth-telling incentivisation methods on survey responses." AEA RCT Registry. AEARCTR-0004860.
https://doi.org/10.1257/rct.4860-3.0
Download:
BibTeX
·
RIS— for reference managers without a browser
translator. Zotero and most
others pick up this page's citation metadata directly.
Diarmaid O Ceallaigh*
, Kirsten I.M. Rohde
, Hans van Kippersluis
Choice-matching is a recently developed method for eliciting honest responses to multiple choice questions in surveys (Cvitanić et al., 2019). Toussaert (2018) has also recently developed a method for incentivising predictions of one's own future behavior which avoids creating incentives to alter actual behavior. We test both of these methods in a general population survey on physical activity preferences and behavior by randomly assigning respondents to one of three versions of the same survey: one which uses choice-matching and the Toussaert method, another which uses choice-matching only, and a third which uses neither method. We test the effect of these methods on survey responses.
Experimental design
We randomly assign respondents to one of three versions of the same survey: one which uses choice-matching and the Toussaert method, another which uses choice-matching only, and a third which uses neither method. Further details are provided in the Hidden Experimental Design which will be visible after the completion of the study.
Design details
Randomization method
Randomization carried out automatically in Qualtrics online survey software.
Randomization unit
Individual
Clusters
N/A
Observations
Invitations to take part in this survey will be sent out to approximately 65,000 members of the Lifelines general population cohort study in the Netherlands. We estimate that approximately 15,000 respondents will complete all three waves of the survey. We estimate that approximately 20,000 will complete wave 1, 16,000 will complete wave 2 and 15,000 will complete wave 3.
1. Number of observations for tests of choice-matching: Estimated 51,000 completed survey responses (20,000 in wave 1, 16,000 in wave 2, 15,000 in wave 3).
2. Number of observations for tests of the Toussaert Method: Estimated 36,000 completed survey responses (20,000 in wave 1, 16,000 in wave 2).
3. Number of observations for test of attrition: Attrition between wave 1 and 2 - Estimated 20,000 respondents. Attrition between wave 2 and 3 - Estimated 16,000 respondents.
4. Number of observations for test of distortion of behavior: Estimated 31,000 completed survey responses (16,000 in wave 2, 15,000 in wave 3)
Arms
1. Tests of choice-matching: Estimated 17,000 survey responses in each of the three arms (treatment A, B and control) (6,666 per arm in wave 1, 5,333 per arm in wave 2, 5,000 per arm in wave 3).
2. Tests of the Toussaert Method: Estimated 12,000 survey responses in each of the three arms (treatment A, B and control) (6,666 per arm in wave 1, 5,333 per arm in wave 2).
3. Test of attrition: Attrition between wave 1 and 2 - Estimated 6,666 respondents per arm. Attrition between wave 2 and 3 - Estimated 5,333 respondents per arm.
4. Test of distortion of behavior: Estimated 10,333 survey responses in each of the three arms (treatment A, B and control) (5,333 per arm in wave 2, 5,000 per arm in wave 3).
Primary outcomes
1. Test of Choice-matching: Responses to a range of questions on physical activity preferences and behavior, as well as questions on monetary preferences, in waves 1-3 of the survey (the survey is longitudinal and has three waves, each approximately 2 weeks apart).
2. Test of Toussaert method: Response to question eliciting predictions of respondents on their physical activity (in hours) over the coming two weeks made in waves 1 and 2 of the survey (predictions of physical activity are not asked in wave 3). Henceforth we call this question the "PA prediction question".
3. Test of attrition: Respondent attrition rates between surveys.
4. Test of distortion of physical activity behavior: Responses to question eliciting self-reported actual number of hours of physical activity over the previous two weeks (henceforth called the "PA actual question"), accuracy of prediction made in the "PA prediction question".
Cited by
No citation data available for this registration yet.