Model uncertainty and overprecision (TAO-RRT)
How to cite
Moore, Don; Backus, Matthew; Augenblick, Ned; Little, Andrew. 2024. "Model uncertainty and overprecision (TAO-RRT)." AEA RCT Registry. AEARCTR-0014396. https://doi.org/10.1257/rct.14396-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Forming beliefs about the world (whether using formal statistical procedures or more generally) usually requires simplifying assumptions. Unfortunately, it is often cognitively costly or even impossible to consider how all possible assumptions would affect the relevant beliefs. We develop a formal model of individuals who properly recognize uncertainty conditional on the assumptions they make (``within-model uncertainty''), but do not recognize how their assumptions affect their beliefs (``across-model uncertainty''). The model's main result is that this leads to overprecision, in the sense of having too low a subjective variance relative to the truth and to the true mean square error of predictions. Making assumptions can also generate disagreement among those with the same information. Further, if assumptions are drawn independently from the correct distribution the amount of disagreement is exactly equal to the amount of overprecision. This study explores these predictions in an experiment that varies within-model and across-model uncertainty.
Intervention
The experimental task presents participants with a problem in which they make a prediction about the mean and variance of an outcome.
In each of 15 rounds, each participant sees a scatterplot image representing sales data over time. The task is to make a prediction about future sales, using two slider bars: one for the mean and the second for the uncertainty around that mean.
Experimental design
The 15 scatterplots are drawn from a set of 160 that vary with respect to: 1) the trendline, 2) noisiness of realizations, 3) whether the true trendline is shown on the image, and 4) the temporal distance to the prediction point from the sample data.
The images participants see are constructed from a set of 40 "problems" each of which has four conditions. A problem is a pattern of dots, and a baseline and alternative distance for the future prediction (which may be shorter or longer). The condition then selects one of the distances, and whether the trendline is given. We allow for 4 possible conditions: 1) a baseline problem (with no line), 2) the baseline with the alternative distance to the prediction point, 3) the baseline with the line depicted, 4) the baseline with both the line depicted and noise scaled such that the correct variance of the prediction is equal to that under condition 1.
Design details
Cited by
No citation data available for this registration yet.