Accounting for Dissenting Evidence

How to cite

Fattinger, Felix; Bossaerts, Peter; Asparouhova, Elena; Yang, Wenhao; Barta, Akos Barnabas. 2025. "Accounting for Dissenting Evidence." AEA RCT Registry. AEARCTR-0016171. https://doi.org/10.1257/rct.16171-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Felix Fattinger* , Peter Bossaerts , Elena Asparouhova , Wenhao Yang , Akos Barnabas Barta

* Primary investigator

Registry ID
AEARCTR-0016171
DOI
https://doi.org/10.1257/rct.16171-1.0
First registered
Last updated
Status
In development
Trial dates
2025-06-09 – 2025-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/16171

Abstract

In this project, we invoke a result from mathematical statistics (Doob, 1949) to identify a curious phenomenon, namely: as it accumulates, dissenting evidence (i.e., evidence against the truth) necessarily becomes more extreme. With the vast majority of evidence eventually supporting the truth, the result is polarization. We design a laboratory experiment that investigates humans’ awareness (ignorance) of this phenomenon and the consequences on their beliefs and demand for additional evidence. Our findings may provide novel insights into social interaction and public opinion formation that are of particular relevance during times of algorithmic and attention-driven news filtering.

Intervention

Our experiment will consist of repeated (independent) individual decision-making tasks. Confronted with two possible scenarios (different two-color compositions of a given urn), participants will be provided with accumulating evidence with respect to two opposing hypotheses (regarding the urn's true majority color). Importantly, participants are fully informed about the probabilistic structure that underlies the evidence accumulation for either hypothesis. We will elicit participants’ beliefs about evidence realizations as well as their willingness to pay for additional evidence. To control for risk preferences, the additional evidence will be provided in two different forms: (1) sample based on one draw vs. (2) sample based on average across 10,000 independent draws (with replacement). Our experimental design entirely abstains from any deception.

Experimental design

The experiment consists of three parts. In Part 1, participants read through detailed instructions, including a visualized demonstration interactively showcasing how to resample from a given urn. At the end of Part 1, participants have to passe a comprehension test in order to proceed. Only if at least 6/8 comprehension questions are answered correctly, participants are allowed to continue to Part 2. If participants fail the comprehension question they are automatically returned to Prolific and only collect the base fee for their time.

In Part 2, the main part of the experiment, participants go through 3 periods, each consisting of three phases. For each period, they are endowed with 2 GBP. In phase 1, they observe the initial sample from the randomly chosen urn and are asked about their beliefs regarding the urn's true majority color. In phase 2, they can decide whether to resample a one-draw sample and if so, what their willingness-to-pay for such a sample is. We elicit participants' willingness-to-pay via a standard Becker–DeGroot–Marschak mechanism (prices ranging from 0 to 1 GBP). In phase 3, they can decided whether to resample an average-draw sample, averaged across 10,000 independent draws, a and if so, what their willingness-to-pay for such a sample is. For both phase 2 and phase 3, the resulting likelihood ratio for the two competing hypotheses (true majority color of the underlying urn) is computed. This likelihood-ratio (capped at 1) determines the chance of winning 3 GBP in a subsequent lottery. At the end, one of the 3 periods is randomly drawn and the two respective lotteries are played out for real.

In Part 3, participants are asked to complete a standard probabilistic reasoning task (6 questions) and fill in a short survey about their educational background. The experiment is implemented with oTree and conducted via Prolific with college-educated participants from the UK and US.

Design details

Randomization method
The randomization is implemented by a computer. The underlying urn selection, the initial samples, and all additional samples are drawn randomly. The only restriction in place is that participants initial sample is informative (excluding samples with 50:50 color distributions). Finally, lottery outcomes are also fully determined randomly adhering to the probabilities implied by participants' choices.
Randomization unit
Randomization (draws from urns and lottery outcomes) occurs at the individual participant level.
Clusters
individual decision-making tasks without further clustering.
Observations
Per participant, we collect 3x2 belief estimates (before and after observing the implied likelihood-ratio), 3 resampling decisions for the one-draw sample (including conditional willingness-to-pay measures), and 3 resampling decisions for the average-draw sample (including conditional willingness-to-pay measures). In total, we are expecting approximately 1,500 observations for each of the above variables.
Arms
Approximately 500 participants (depending on budget restrictions implied by random payments)
Primary outcomes
We focus on particiapnts' beliefs about competing hypotheses and their willingness to pay for additional evidence. A main control will consist of their performance in a standardized probabilistic reasoning task.

Cited by

No citation data available for this registration yet.

Keywords