We study how people solve statistical problems. We test a model in which selective attention to different features of problems yields multimodality in beliefs. Our experiments are designed to change the distribution of participants beliefs by manipulating the contrast or prominence of various features while (typically) holding constant the underlying statistical problem.
Experimental design
The experiment has two main questions—one inference problem and one gambler's fallacy problem—that occur in a random order at the beginning of the experiment. After each of them, we elicit self-reports of how participants solved each question as well as what features of the problem they felt they were paying attention to.
Design details
Randomization method
All randomization will be done within Qualtrics using either its survey flow randomizer or javascript embedded in the survey.
Randomization unit
Treatment assignment is randomized at the individual level.
Clusters
We are randomizing at the individual level, so there are no clusters.
Observations
We hope to recruit 4800 participants through prolific.
In addition, in our April 2024 experiments, we will recruit another 3,000 participants.
Arms
Intended sample sizes for each inference treatment are below:
Treatment 1: 500
Treatment 2: 200
Treatment 3: 200
Treatment 4: 200
Treatment 5: 500
Treatment 6: 500
Treatment 7: 200
Treatment 8: 200
Treatment 9: 500
Treatment 10: 500
Treatment 11: 500
Treatment 12: 200
Treatment 13: 200
Treatment 14: 200
Treatment 15: 200
For the gambler's fallacy treatments, intended sample sizes are as follows:
Treatment 1: 400
Treatment 2: 400
Treatment 3: 1000
Treatment 4: 1000
Treatment 5: 1000
Treatment 6: 1000
Treatment is independently randomly assigned, so final sample sizes may differ somewhat from the above numbers due to chance.
See above for sample sizes in our April and May 2024 experiments.
Primary outcomes
Our primary questions of interest are variants on the "balls-and-urns" inference paradigm and on questions about coin flips ("Gambler's Fallacy" problems).
Secondary outcomes
We will also elicit from participants free-text responses of how they solved each problem and self-reports of which features they felt they were paying attention to. Next, we will have participants rate the similarity between pairs of sequences of coin flips, to back out which features they attend to, as well as their beliefs about the absolute frequency of various sequences. We plan to correlate judgments of average similarity with these frequency judgments.
Cited by
No citation data available for this registration yet.