Predicting experimental results: The role of incentives, anchoring, and experience

How to cite

Lergetporer, Philipp; Ruthardt, Fabian; Potrafke, Niklas; Gründler, Klaus; Grewenig, Elisabeth. 2020. "Predicting experimental results: The role of incentives, anchoring, and experience." AEA RCT Registry. AEARCTR-0005944. https://doi.org/10.1257/rct.5944-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Philipp Lergetporer* , Fabian Ruthardt , Niklas Potrafke , Klaus Gründler , Elisabeth Grewenig

* Primary investigator

Registry ID
AEARCTR-0005944
DOI
https://doi.org/10.1257/rct.5944-1.0
First registered
Last updated
Status
In development
Trial dates
2020-06-03 – 2021-12-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/5944

Abstract

We let people predict the outcome of an online survey experiment and investigate the causal effects of (i) monetary incentives, (ii) anchoring, and (iii) participants’ experience in the experiment on the accuracy of predictions. Therefore, we implement an online-survey experiment among a representative sample of adults aged 18 to 69 years in Germany, and additionally survey a sample of experts (economics professors).
Respondents in the representative sample are randomized into six different experimental groups. The first three groups participate in an “information experiment” on preferences for increased school spending. Groups 4, 5, and 6 participate in a “prediction experiment” and predict the results from the “information experiment”. Group 4 receives no incentives for a correct prediction and no anchor when stating the prediction. Group 5 is offered a monetary incentive for a correct prediction, and group 6 is provided with an anchor. In addition, we study the causal effect of experience on prediction accuracy by letting groups 1, 2, and 3 predict the results in the “information experiment” after participating in that experiment. The expert sample completes the same prediction task as group 4.
Comparing prediction accuracy between experts and non-experts, we investigate whether providing incentives, anchors, and experience to non-experts improves prediction accuracy of the general population towards the levels of experts.

Intervention

We let subjects predict the outcome of an online survey experiment and investigate the causal effects of (i) providing monetary incentives for correct predictions, (ii) providing an anchor of control-group behavior in the experiment, and (iii) participating in the experiment on the accuracy of subjects’ predictions. Therefore, we implement an online-survey experiment among a representative sample of adults aged 18 to 69 years in Germany, and additionally survey a sample of experts (economics professors) from a regularly conducted German expert sample.
Respondents in the representative sample are randomized into six different experimental groups. The first three groups participate in an “information experiment” on preferences for increased school spending. Group 1 receives no information when stating spending preferences; group 2 is informed about average current school spending per student, and respondents in group 3 have the option to acquire the spending-information provided to group 2. Groups 4, 5, and 6 participate in and “prediction experiment” and predict the results of the “information experiment”. Group 4 receives no anchor and no incentive when stating the prediction. Group 5 is offered a monetary incentive for a correct prediction, and group 6 is provided with an anchor. In addition, we study the causal effect of experience on prediction accuracy by letting groups 1, 2, and 3 predict the results in the “information experiment” after participating in that experiment. The experts complete the same prediction task as group 4. Comparing prediction accuracy between experts and non-experts, we investigate whether providing incentives, anchors, and experience to non-experts improves prediction accuracy of the general population towards the levels of experts.

Experimental design

We conduct the experiment in a sample of 6,000 adults aged between 18 and 69 years. The survey is conducted in cooperation with respondi. The recruitment and polling is managed by respondi, who collect the data via an online platform. That is, our participants answer the survey questions autonomously on their own digital devices. Randomization is carried out by respondi at the individual level, using a computer. In addition, the expert sample from the “ifo Ökonomenpanel” will cover around 200 economics professors in Germany.

Our experiment in the representative sample is structured as follows:
Treatment 1:
Stage 1: prior beliefs on average school spending
Stage 2: preferences for school-spending increases (no information)
Stage 3: predicting experimental results
Stage 4: open-ended question on prediction formation
Stage 5: closed-ended question on prediction formation

Treatment 2:
Stage 1: prior beliefs on average school spending
Stage 2: preferences for school-spending increases (information provision)
Stage 3: predicting experimental results
Stage 4: open-ended question on prediction formation
Stage 5: closed-ended question on prediction formation

Treatment 3:
Stage 1: prior beliefs on average school spending
Stage 2: preferences for school-spending increases (voluntary information acquisition)
Stage 3: predicting experimental results
Stage 4: open-ended question on prediction formation
Stage 5: closed-ended question on prediction formation

Treatment 4:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation

Treatment 5:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results (with incentives)
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation

Treatment 6:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results (with anchor)
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation

Our design in the expert sample is structured as follows:
Stage 1: prior beliefs on average school spending
Stage 2: predicting experimental results (from the representative sample)
Stage 3: open-ended question on prediction formation
Stage 4: closed-ended question on prediction formation

Design details

Randomization method
Randomization is carried out by the survey company respondi, using a computer.
Randomization unit
at the individual level
Clusters
6,000 adults aged 18 – 69 years + 200 experts (economics professors)
Observations
6,000 adults aged 18 – 69 years + 200 experts (economics professors)
Arms
6,000 adults aged 18 – 69 years, approx. 1,000 will be assigned to each of the treatment groups. 200 experts (economics professors)
Primary outcomes
Our primary outcomes of interest are respondents’ predictions of average answers in the three groups of the “information experiment”.
Secondary outcomes
Treatment-effect heterogeneities in the “prediction experiment” by prior beliefs (elicited in stage 1), and respondents’ highest educational degree.

Cited by

No citation data available for this registration yet.

Keywords