Using Response Times in Surveys, Study 2
How to cite
Garagnani, Michele; Alos-Ferrer, Carlos. 2023. "Using Response Times in Surveys, Study 2." AEA RCT Registry. AEARCTR-0012668. https://doi.org/10.1257/rct.12668-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Participants are recruited online using Prolific. They are asked to perform a series of decision tasks in exchange for a flat payment and additional decision-dependent incentives in part of the tasks. In particular, participants are asked to rate 20 charities and organizations according to how much they favor them using 9-point Likert scales (see the supporting document). Two additional questions after the ratings ask for the most- and least-favorite organization from the list. Participants are then asked to give binary answers to several survey questions (see the supporting document). The order of ratings and questions will be randomized for each subject (within blocks of similar questions). We
will further measure response times for each binary choice. We plan to compare how the answers to these questions might differ between groups of participants (see the supporting document for how we pre-define the groups).
Intervention
Participants are recruited online using Prolific. They are asked to perform a series of decision tasks in exchange for a flat payment and additional decision-dependent incentives in part of the tasks. In particular, participants are asked to rate 20 charities and organizations according to how much they favor them using 9-point Likert scales (see the supporting document). Two additional questions after the ratings ask for the most- and least-favorite organization from the list. Participants are then asked to give binary answers to several survey questions (see the supporting document). The order of ratings and questions will be randomized for each subject (within blocks of similar questions). We
will further measure response times for each binary choice. We plan to compare how the answers to these questions might differ between groups of participants (see the supporting document for how we pre-define the groups).
Experimental design
Charities to be rated are selected because they are either popular or unpopular, according to yougov.uk for Q3-2023. The popular charities we include had approval rates between 73% and 86% (this was the overall maximum) at the time of the design of the study (November 2023), and a “Fame” of 90% or above: British Heart Foundation; Macmillan Cancer Support; Samaritans; WWF; British Red Cross; Guide Dogs; RSPB (Royal Society for the Protection of Birds). We tried to include a variety of themes, e.g. avoiding having two different cancer charities on the list. Less popular charities had approval rates between 24% and 60%: UNICEF; Fairtrade Foundation; Amnesty International; Cats Protection; Christian Aid; Greenpeace; Unite to End Violence Against Women; Black Lives Matter; PETA; Campaign for Nuclear Disarmament; Vegetarian Society.
We also include the two main political organizations in the UK, the Conservative Party and the Labour Party.
We also ask 38 (20+10+4+4) binary questions which are of 3 different kinds:
Type 1. We ask participants to decide how they want to split an amount of money (4 Pounds) between themselves and a charity. In particular, the question asks participants to decide whether they would prefer to keep the entire amount or split the sum evenly (50% - 50%) with the charity. We ask this question for all 20 charities and organizations participants rated.
For each participant, one of the questions of type 1 is randomly selected and paid at the end of the survey. That is, the participant’s decision is actually implemented for the selected question (including making the donation to the corresponding organization). Randomization will be made at the charity level, i.e. we will pick one of the 20 questions at random and realize the corresponding question for all participants.
Type 2. We present participants with two lines of different lengths, one above the other. We ask them if the one above is longer than than the other. We repeat this question for different lengths 10 times. For each participant, one of the questions of type 2 is randomly selected and paid at the end of the survey. If the participant gave the correct answer, he or she will earn an additional one pound.
Type 3. We ask participants to answer four framing and invariance questions analogous to the ones proposed by Tversky and Kahneman (1981, 1986). Participants are randomly assigned to see only one of the two versions of the questions (e.g., either gain frame or loss frame).
Questions of type 3 are not incentivized.
Design details
Cited by
No citation data available for this registration yet.