Prolific sample for: Competing Causal Interpretations: An Experimental Study
How to cite
Ambuehl, Sandro; Thysen, Heidi. 2023. "Prolific sample for: Competing Causal Interpretations: An Experimental Study." AEA RCT Registry. AEARCTR-0012652. https://doi.org/10.1257/rct.12652-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This is a follow up to a lab study we conducted previously. The abstract of the lab study is this:
A central factor when choosing an action is its causal effect on the outcome of interest. Yet, causal information is often lacking. People instead observe correlational or historical data, along with causal interpretations and recommendations provided by experts who frequently disagree with each other. Our laboratory experiments study choice in such settings, where beliefs concern the structure of the data-generating process rather than merely magnitudes. Roughly half of our subjects attempt to determine the fit of the causal interpretations to past data, as the literature on model persuasion assumes. We characterize the limits to their ability to do so. Half the subjects’ choices are co-determined by the interpretations’ promises of future payouts, as the literature on narrative competition assumes, or by the downside these choices entail if they are mistaken. Subjects also commonly employ heuristics such as Occam’s razor. The fact that they typically prefer flexibility over parsimony insures them against bad choices in some settings but has the opposite effect in others. Our estimates predict well out of sample and closely agree across two different identification strategies in two different samples. We also study the extent to which behavior is robust to framing and the relation between subjects’ choices and their political attitudes and psychological characteristics. Our results characterize the cases in which subjects’ behavioral tendencies render them most receptive to misleading interpretations. They also inform the literatures on narrative competition and model persuasion.
Intervention
SETTING: See description on experimental design
INTERVENTION: We run two treatments. In one treatment, we provide help to subjects, in two forms. First, subjects have access to a description of the correlational implications of causal structures. Second, we highlight data charts for which the competing interpretations have conflicting implications. In the other treatment, subjects get no such help.
Experimental design
In our experiment, subjects choose an action at a cost. The action possibly affects a monetary outcome. Whether it does and by how much depends on a data-generating process (DGP) that involves the action, the outcome, and an additional quantity (the covariate). Subjects do not know the DGP, but they observe information about it. Specifically, in each round, we fit a misspecified model to large datasets generated by the DGP, along with the true model. For each model, we calculate the implied optimal action (the recommendation), and the payoff the subject can expect if the specification is correct and the subject follows that model (the promise). Subjects observe each model’s recommendation and promise. While we do not tell subjects which models are misspecified, we present them with a graphical description of each model in the form of a directed acyclic graph (DAG). In addition, subjects can observe unconditional correlations between any pair of variables in the data generated by the DGP, as well as the correlations between any pair of variables conditional on any third variable. Subjects who wish and are able to do so can therefore check which models may be misspecified. Throughout, our DGPs and misspecified models are recursive systems of linear Gaussian equations. Subjects choose which model to follow. The corresponding action recommendation is carried out and determines the subjects’ payout according to the DGP.
We frame this setting to subjects as follows. A true ‘mechanism’ determines the relation between the subject’s ‘action’ and her ‘bonus.’ Two or three ‘advisors’ each have a ‘theory’ about that mechanism. While data cannot affect an advisor’s theory, each advisor has used the data generated by the true mechanism to derive a recommendation for what action is optimal and what bonus the subject can expect if the advisor’s theory is correct. Subjects select one of the advisors whose recommendation is carried out.
Design details
Cited by
No citation data available for this registration yet.