Simplicity vs Precision in Eliciting Probabilistic Expectations

How to cite

Pavlova, Lora. 2020. "Simplicity vs Precision in Eliciting Probabilistic Expectations." AEA RCT Registry. AEARCTR-0006482. https://doi.org/10.1257/rct.6482-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Lora Pavlova*

* Primary investigator

Registry ID
AEARCTR-0006482
DOI
https://doi.org/10.1257/rct.6482-2.0
First registered
Last updated
Status
Completed
Trial dates
2020-09-17 – 2020-09-30
Countries
Germany ()
Registry record
http://www.socialscienceregistry.org/trials/6482

Abstract

The practice of directly surveying consumers' macroeconomic expectations has become more widespread in recent years, as the generated data finds broader applications in economics. In particular, growing attention is paid to probabilistic expectations, which provide informative measures of individual uncertainty. However, there is still no clear consensus on the best question wording and format, despite them being central in the process of eliciting probabilistic expectations. Existing research states that survey questions, as a form of communication should ultimately be understood and interpreted in the same way by both researchers and participants. Complex, hard-to read questions that cause high non-response rates are therefore undesirable. This applies equally to question format: validity of results using expectations data strongly rely on both respondents' willingness and ability to express expectations over uncertain events in a probabilistic form. Ideally, the question format should take both into account. To sum up, a question should combine a simple wording and design, such that the elicited expectations accurately represent the respondents beliefs about the future outcome. On the other hand, simplified wordings often involve a certain degree of ambiguity and lay the ground for diffuse interpretations such that the resulting answers are no more interpersonally comparable. I address the trade-off between simplicity and precision in survey design in the context of inflation expectations of a representative sample of German consumers.

In the current experiment, conducted as part of the Bundesbank Online Pilot on Consumer Expectations, I test the effects of variations in question wording and format on consumer response behaviour. To this end, respondents are broken down into four treatment groups and presented versions of the question about probabilistic expectations. As part of the experiment, two competing wording versions, previously known from leading consumer surveys, are considered in the treatments: expected change in (i) prices in general or (ii) the inflation rate. In addition, a question format to elicit a subjective distribution alternative to the one already widely used is also included in the treatments. So far, the practice has been for respondents to assign probabilities that the inflation rate over the next 12 months will fall within a certain range. As opposed to this, in two of the four treatments a less restrictive setting is used, where survey participants are asked to report the most likely, the minimum and the maximum outcome they expect. I hypothesize that the 'inflation-rate' treatment, while prompting people to think more about price changes on a broader scale, also generates a higher non-response rate, especially among the less educated population. I also predict that moving from a probabilistic question formulation with predefined intervals to a less restrictive one, one should observe an upward shift in expectations coupled with larger variation in responses.

Experimental design

Respondents are randomly assigned to four groups and presented versions of the question about probabilistic expectations, where both wording and format are varied. Then respondents are asked a follow-up quesion on how they intepreted the wording of the previous quesions about their expectations.

Design details

Randomization method
Randomization is done by a survey firm.
Randomization unit
Individual.
Clusters
4000 respondents randomly assinged to four groups.
Observations
4000 respondents
Arms
Approximately 1000 in each of the four treatment arms.
Primary outcomes
Effects of question wording and format on consumer response behaviour. In particular, patterns of assigning probability to intervals, presence of outliers, variance and IQR of the subjective distribution, disagreement among respondents, non-response rate, internal consistency of answeres and heterogeniety of the effects with respect to demographic groups.

Cited by

No citation data available for this registration yet.

Keywords