Unincentivized surveys of stated-preference may be susceptible to "socially desirable responding" (SDR, hereafter) where respondents strategically conceal preferences that they perceive to be socially undesirable. In contrast, economists often assume that the incentive-compatibility of revealed-preference elicitations mitigates the threat of SDR by imposing real costs. One domain in which SDR appears particularly powerful is the realm of politics where SDR is both ubiquitous and well-known by the popular press and other "consumers" of polling research. Moreover, incentive-compatible elicitations of political preferences are not feasible. However, if awareness of SDR is sufficient to de-bias polling information, then SDR may not cause any negative consequences. This project seeks to test the ability of economic agents to de-bias polling data subject to SDR.
In an experimental framework, we will test the accuracy of subjects tasked with predicting a series of choices potentially subject to SDR. Our experiment will have two treatments. In Treatment 1, subjects will observe unincentivized, stated-preference data. In Treatment 2, subjects will observe incentivized, revealed-preference data. While Treatment 1 will be subject to both SDR and sampling errors, Treatment 2 will only be subject to sampling errors. Thus, differences in accuracy will be attributable to differences in the magnitude of bias from SDR. With perfect sophistication, subjects in the two treatments should be equally accurate. However, with partial or fully naive subjects, predictions from Treatment 1 will manifest some of the SDR from the underlying data and will be less accurate. We will measure accuracy in terms of the magnitude and direction of the prediction errors.
Intervention
Subjects in our experiment will be making predictions about a series of choices potentially subject to SDR. Different subjects will receive different information prior to making their predictions.
Experimental design
Subjects in our experiment will be making predictions about a series of choices potentially subject to SDR. Different subjects will receive different information prior to making their predictions.
Design details
Randomization method
To divide subjects into the Hypothetical and Incentivized Choice groups, we will randomly divide our subject pool into two parts and recruit subjects for the groups separately. Recruitment will be identical for the two groups. Subjects will enroll through the SONA recruitment system.
At the prediction stage, Qualtrics will randomly assign subjects to receive information based on the Hypothetical or Incentivized Choice groups. We will block "Predictors" based on their prior experience (Hypothetical Choice group, Incentivized Choice group, or neither) This process will be mechanically random and balanced in sample sizes.
Randomization unit
Each subject will be randomly assigned to receive information from the Hypothetical or Incentivized choice groups.
Clusters
800 subjects
Observations
300 subjects (with 1000 extra in a planned extension)--
100 Hypothetical Choice subjects from the University of Arkansas
100 Incentivized Choice subjects from the University of Arkansas
100 new subjects from the University of Arkansas who make predictions only
(planned extension) 1000 subjects from Amazon's Mechanical Turk who make predictions only
Arms
150 subjects receiving information from the Hypothetical Choice group.
150 subjects receiving information from the Incentivized Choice group.
(Planned extension)
500 subjects receiving information from the Hypothetical Choice group.
500 subjects receiving information from the Incentivized Choice group.
Primary outcomes
Our primary outcomes of interest are:
The accuracy of the final predictions: Both the direction and magnitude of the difference between the prediction (Prior or Posterior) and the truth.
The weight placed on each source of information: How much subjects update their predictions based on the information received.
We will look at the differences in each of these outcomes based on:
Receiving information from the Hypothetical or Incentivized Choice groups
Whether the information you received was in the socially-desirable direction
The interaction of information source and social-desirability.
Being previously assigned to the Hypothetical Choice group, Incentivized Choice group, neither group (but from the University of Arkansas), or neither group (recruited from MTurk)
Whether the information you received was from the treatment to which you were assigned
Secondary outcomes
We will also evaluate unincentivized predictions about actions that may be subject to SDR, but we will not observe directly (e.g. unsafe sex). These predictions will be part of a calibration exercise where we use the observed ability of subjects to de-bias information from the Hypothetical choice group to improve predictions about actions subject to SDR.
Cited by
No citation data available for this registration yet.