Underconfidence or lottery design? – A replication of Hoelzl & Rustichini (2005)

How to cite

Protte, Marius. 2024. "Underconfidence or lottery design? – A replication of Hoelzl & Rustichini (2005)." AEA RCT Registry. AEARCTR-0013070. https://doi.org/10.1257/rct.13070-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Marius Protte*

* Primary investigator

Registry ID
AEARCTR-0013070
DOI
https://doi.org/10.1257/rct.13070-1.0
First registered
Last updated
Status
In development
Trial dates
2024-06-01 – 2024-09-30
Countries
Private
JEL codes
C91, D91, D71
Registry record
http://www.socialscienceregistry.org/trials/13070

Abstract

The 2005 paper “Overconfident: Do you put your money on it?” by Eric Hoelzl and Aldo Rustichini introduces a novel approach to studying of overconfidence in comparison to others. By letting participants vote for a performance test or a lottery as the basis for a bonus payment, they present a behavioral measure for relative overconfidence that goes beyond verbal performance self-assessments. They observe a tendency for underconfident assessments under monetary incentives with hard test items, i.e. individuals increasingly opting for a luck-based payment scheme (die roll) over the performance-based one (knowledge test), presumably due to individuals perceiving their chances to be higher in a fair 50/50 lottery than in a (difficult) performance test.
More recent literature has noted that the observed voting patterns do not necessarily constitute underconfidence but may rather be attributed to attitudes of aversion against risk or ambiguity associated with the test (Grieco & Hogarth 2009, Blavatskyy 2008, Owens et al. 2014). We further argue that the two payment schemes are not perfectly comparable in terms of perceived winning probability, as the test presents a fixed winner distribution of 50% (i.e. no variance in the amount of winners), while the lottery presents a probabilistic winner distribution with a variance to it. This inherent variance may bias the voters’ perception of their probability to win and may even lead to altruistic behavior in the sense that “everybody has the chance to win on an individual die roll”, in which the chance of winning is not interdependent with other participants performance. For these reasons, we suspect that the voting share in favor of the lottery that is due to underconfidence may be overestimated.
Therefore, in the planned experiment, a fixed outcome distribution lottery (50/50) will be compared against the probabilistic 50/50 lottery from Hoelzl & Rustichini (2005) in a between-subject design to evaluate the impact of the lottery mechanism’s design on individual voting behavior. Additional questionnaire measures will be implemented to gain a more complete understanding of the driving motive(s) behind individual voting behavior to either rebut or affirm alternative explanations besides underconfidence.

Intervention

In the planned experiment, a fixed outcome distribution lottery with a 50% win probability will be compared against the probabilistic 50% win probability lottery from Hoelzl & Rustichini (2005) in a between-subject design to evaluate the impact of the lottery mechanism’s design on individual voting behavior.
Additional questionnaire measures will be implemented to gain a more complete understanding of the driving motive(s) behind individual voting behavior to either rebut or affirm alternative explanations besides underconfidence.

Experimental design

The experiment by Hoelzl & Rustichini (2005) will be replicated in an online experiment on Prolific. Subjects will vote on a performance-based (test) or luck-based (lottery) payoff scheme for a potential bonus payment, which serves as an indicator for accurate self-assessment relative to others. Which payoff mechanism is eventually implemented is determined by the majority vote among all 100 participants. Independent of voting outcomes, all subjects will play both the test and the lottery afterwards, to prevent effort avoidance. The performance test will be based on vocabulary knowledge and consists of 20 items. The lottery mechanism will differ between the replication condition (probabilistic outcome distribution lottery) and the treatment condition (fixed outcome distribution lottery). Before and after the performance test, participants will be asked the same questions on their pre-test expectations and post-test reflection as in the original study by Hoelzl and Rustichini. Ultimately, subjects will answer a multi-page questionnaire like attitudes in terms of risk, ambiguity, altruism, social comparison, and self-efficacy to elicit alternative voting motives.

Design details

Randomization method
Public study on Prolific.com
Randomization unit
Not applicable
Clusters
200 participants from Prolific.com
Observations
200 participants from Prolific.com
Arms
100 participants in the replication (control) group, 100 participants in the adaptation (treatment) group
Primary outcomes
Individual voting behavior compared between probabilistic distribution and fixed outcome distribution lottery schemes

Cited by

No citation data available for this registration yet.

Keywords