Strategic Communication and the Underuse of Incentive-Based Credibility: Experimental Evidence

How to cite

Ressi, Anna; Feess, Eberhard; Jost, Peter-J.. 2023. "Strategic Communication and the Underuse of Incentive-Based Credibility: Experimental Evidence." AEA RCT Registry. AEARCTR-0010984. https://doi.org/10.1257/rct.10984-4.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Anna Ressi* , Eberhard Feess , Peter-J. Jost

* Primary investigator

Registry ID
AEARCTR-0010984
DOI
https://doi.org/10.1257/rct.10984-4.0
First registered
Last updated
Status
Completed
Trial dates
2025-06-26 – 2025-08-31
Countries
United States of America ()
JEL codes
C91; D83; D84; D91
Registry record
http://www.socialscienceregistry.org/trials/10984

Abstract

In many economically important environments, people receive information from sources whose interests are transparent. Such communication naturally invites skepticism because receivers understand that senders have incentives to influence their beliefs. However, skepticism should not be indiscriminate. When a sender communicates a message that contradicts the direction in which the sender benefits from influencing the receiver, the message becomes particularly credible because it is difficult to reconcile with purely opportunistic behavior.
In this paper, we ask whether receivers exploit this credibility of such incentive-incongruent messages or whether they continue to rely primarily on their own judgment, even when sender incentives imply that a message is unusually likely to be true?
Participants in our experiment act as receivers. They first answer factual questions comparing outcomes under the Obama and Trump administrations. They then observe a message from a sender in a previous study. Senders knew the correct answer, could send either possible message, and had a monetary incentive to induce either an "Obama" or a "Trump" answer. Receivers know the sender's incentive and earn a bonus for choosing the correct answer. This setup allows us to examine how receivers respond to messages, depending on the senders’ incentives and their own prior beliefs.
In additional treatments, we either elicit participants’ estimates of how frequently senders lie or provide them with explicit information on this frequency. This allows us to examine how underreaction to credible messages can be explained and mitigated.

Experimental design

Participants in the role of receivers know that there are two main stages of which only one will be randomly chosen for determining their bonus.
In stage 1 of the experiment, we randomly select two out of four questions. For half of the questions, the correct answer is “Obama” and for the other half, the correct answer is “Trump”. Specifically, participants are asked for their belief on whether the unemployment rate in a particular US state, and the crime rate in another US state, was lower during the Obama or the Trump administration. The two questions are presented in a random order. Participants know that, given that stage 1 is paid out, exactly one of the two questions will be randomly chosen to determine their payoff and that they will receive a bonus if and only if their answer to this question is correct. For each question, we also ask them how certain they are that their belief is correct.
In stage 2, receivers can revise their previous answers based on a message of a sender in a previous study. For those messages, we use data from a previous study. Hence, all participants in our new experiment are receivers. In this previous study, we informed senders about the correct answer. For each question, they could then send one out of two possible messages, suggesting that the correct answer is “Obama” or “Trump”. Each of the two messages was sent to a receiver of the previous study. Senders in treatment TRUMP (OBAMA) knew that they would receive a bonus if and only if the receiver’s answered “Trump” (“Obama”).
Receivers are informed that senders knew the correct answers but did not need to report truthfully. We also inform them about the senders’ incentives. After they receive the senders’ messages, we again ask them whether they believe that “Obama” or “Trump” is the correct answer. Receivers know that, given that stage 2 is paid out, they will get a bonus if and only if their answer is correct.
To elicit the receivers’ responses to the senders’ messages, we use the strategy-method: For each of the two questions, receivers have to choose between “Obama” and “Trump” for each of the two possible messages of the sender. Payoffs are calculated based on the message the sender actually sent.
In the post-experimental questionnaire, we elicit basic demographic information and personal attitudes, such as their political affiliation.
Overall, our first treatment manipulation varies the senders’ incentives:
o In treatment OBAMA, receivers know that the senders got a bonus if and only if the receiver they were matched with announces that “Obama” is the correct answer.
o In treatment TRUMP, receivers know that the senders got a bonus if and only if the receiver they were matched with announces that “Trump” is the correct answer.

We also have the following additional treatment variations:
o Treatment A1 & A2: We ask receivers the following incentivized questions before (treatment A1) or after (treatment A2) they provide their final answers:
• If (TRUMP / OBAMA) is the correct answer, how many out of 100 senders sent the true message (TRUMP / OBAMA).
• If (OBAMA / TRUMP) is the correct answer, how many out of 100 senders sent the false message (TRUMP / OBAMA).
For each of the two questions, they will get an additional bonus of £0.10 if their guess is less than 5 percentage points away from the true share.
o Treatment B: We give receivers the following information before they provide their final answers
• If (TRUMP / OBAMA) is the correct answer, (95 / 96) out of 100 senders sent the true message (TRUMP / OBAMA).
• If (OBAMA / TRUMP) is the correct answer, (52 / 63) out of 100 senders sent the false message (TRUMP / OBAMA).
o Treatment C: We give receivers the following information before they provide their final answers:
• Out of 100 messages (TRUMP / OBAMA), (66 / 62) were true.
• Out of 100 messages (OBAMA / TRUMP), (89 / 90) were true.
o Treatment D: Before receivers provide their final answers, we ask them to imagine that they are in the role of the sender and ask them the following two questions:
• Suppose (TRUMP / OBAMA) is correct. Which message would you send?
• Suppose (OBAMA / TRUMP) is correct. Which message would you send?
As in treatment A2, we also elicit their estimated lying frequencies after they provide their final answer.

We use a between-subject design: Each subject participates in only one treatment. The experiment will be conducted online using the survey software Qualtrics and participants will be recruited using Prolific. We will require participants to reside in the US and show an approval rate of at least 95%. Participants who answered the majority of our comprehension questions incorrectly will not be allowed to continue and will hence not be included our sample. Moreover, in the post-experimental questionnaire, participants are asked whether they looked up the answers to the estimation questions on the internet. Those who respond “Yes” will be excluded from the statistical analysis.

Design details

Randomization method
All randomization procedures during the study—whether assigning participants to the TRUMP or OBAMA treatments or randomizing the order of questions and answer options—are implemented by the software.
Randomization unit
Individual level randomization
Clusters
We cluster on the participant level.
Observations
We plan to collect 2,400 observations (usable for our statistical analysis) in total – 600observations in each of our four treatments. To obtain our planned number of observations, we need to recruit 300 receivers for each treatment (150 in treatment TRUMP and 150 in treatment OBAMA), i.e., 1,200 participants in total. This is because our use of the strategy method for eliciting receivers’ belief implies that we obtain 4 observations per participant (two messages for two questions), i.e., 4,800 observations in total. Since our analyses will focus on those receivers for whom the initial answer is not in line with the message they receive, only half of the observations, i.e. 2,400, will actually be used.
Arms
Because we elicit each participant’s decisions for both possible messages in both domains, we need to recruit a total of 1,200 participants – 300 in each of our four treatments A1/A2, B, C, and D (150 per treatment OBAMA and TRUMP).
Primary outcomes
We test the predictions S1-S2 for senders and R1-R4 for receivers as stated in the previous version of our pre-registration: - Hypotheses for senders S1. Incentives: The probability of lying is lower if the truth corresponds to the senders’ incentives than if the truth does not correspond to the senders’ incentives. S2. Lying costs: The probability of lying is higher the lower the senders’ lying costs. - Hypotheses for receivers R1. Partisanship I: The probability that the initial answer is "Trump administration" ("Obama administration") is higher for Republicans (Democrats). R2. Partisanship II: The probability that receivers who change their initial answer from “Trump administration” to “Obama administration” (“Obama administration” to “Trump administration”) is higher for Democrats (Republicans). R3. Senders’ incentives: The probability that receivers change their initial answer is lower if the message corresponds to the senders’ incentives than when it does not correspond to the senders’ incentives. R4. Belief: The probability that receivers change their initial answer decreases with the strength of their belief in stage 1. These predictions are derived from our (updated) game-theoretic model. Additionally, we now test our previously defined secondary outcome S3 as a main outcome, as it also follows from the model. We no longer test R5 from the previous version of our pre-registration because we dropped the associated treatment manipulation (receivers’ first-stage answers incentivized vs. not incentivized) from our new design. Based on the data collected in the earlier version of the study, we additionally test the following hypotheses: - H1: Across all treatments (A1/A2, B, C, D), we predict that the frequency of type-II errors exceeds that of type-I errors. This pattern is expected to hold regardless of whether the message is aligned or misaligned with the sender’s incentives, as well as whether the message is aligned or misaligned with the receiver’s partisanship. - H2: The difference between type-II and type-I errors is greatest in treatments A, followed by treatment B and then treatment C. - H3: The difference between type-II and type-I errors decreases in treatment D compared to treatments A. - H4: In treatments A1, A2 and D, the probability that a receiver switches increases with their belief about the senders’ lying costs. For calculating the frequency of type-I and type-II errors, we derive each receiver’s subjectively optimal decision according to their ex-ante belief that their answer is correct and, depending on the treatment and the receiver’s information, the estimated or actual lying frequencies of the senders. We test these hypotheses using both non-parametric and parametric methods (regression analysis). As in the previous version of the pre-registration (and consistent with the predictions of our game-theoretic model), our analyses will focus on observations for which the sender’s message is not aligned with the receiver’s initial answer.
Secondary outcomes
There are two kinds of additional secondary outcomes for which our study is explorative: 1. We examine whether receivers earn higher payoffs when switching from their initial answer, depending on whether the sender’s message is or is not in line with their incentives. 2. We investigate the influence of our control variables (for demographics mainly age and gender) as well as personal and political attitudes. We also explore the impact of our two question domains (crime and unemployment). Because our previously collected data did not reveal statistically significant differences between treatments in which the receiver’s stage-1 answers were and were not incentivized, the new design omits this treatment variation. Therefore, we no longer include the related secondary outcomes listed in the earlier version of the pre-registration.

Cited by

No citation data available for this registration yet.

Keywords