Persistent Stories or Persistent Numbers: Effects of Narrative versus Statistical Evidence on Policy Preferences
How to cite
Nakabayashi, Masaki; Kawata, Keisuke; McElwain, Kenneth. 2022. "Persistent Stories or Persistent Numbers: Effects of Narrative versus Statistical Evidence on Policy Preferences." AEA RCT Registry. AEARCTR-0008821. https://doi.org/10.1257/rct.8821-1.2
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
1. Motivation
Many research studies have shown that intervention effects are short-lived. However, we consider that the short life of a treatment effect originates to a substantial degree a uni-dimensional experimental setting, such as requiring greater efforts to save energy or towels, power, or water. In fact, many essential issues of our lives do not involve uni-dimensional decision-making. When a legislature approves a budget for the coming year, it does not necessarily increase or decrease the taxation rate. Instead, it discusses a possible budget vector while viewing the scholar of the vector, or the revenue, as given. Improvement in public policy does not necessarily mean a larger size of the government but instead could mean wiser spending given a constant revenue. Traditional uni-dimensional design implicitly assumes additional effort or tax payments and might discourage long-term improvement in behavior or spending.
To address this issue, we conceive a multidimensional public policy space in which hypothetical policy packages are randomly generated by a fully randomized conjoint design. The policy space contains a possible shrinking of government size as well as spending for public policies. If a respondent wants to keep the size of government constant or even shrink it, she or he still might want to relieve poverty by reallocating spending in other directions. By allowing for such choices, we can accurately identify treatment effects on policy preferences.
Additionally, possibly different impacts of different modes of information have been discussed. We use both statistical evidence and narratives to exchange findings and ideas on a daily decision-making basis. While we implicitly believe that statistical or narrative evidence might be more effective under specific circumstances, empirical results remain mixed. We estimate the possible difference in choices within our hypothetical public policy space.
2. Objective
We investigate (1) whether information provision affects public policy preferences in the long term by a three-wave panel conjoint experiment performed from March 2022 to September 2022 and to February 2023. If we find that information provision does affect policy preferences, we investigate (2) whether narrative or statistical evidence affects policy preferences stronger/longer. Rom among several critical issues, we adopt poverty relief as a focus and study the relative preference for poverty relief compared with other dimensions of public policy; public works, education, public health care, the public pension fund, and fiscal retrenchment.
3. Setting and design
We conduct a panel internet survey by recruiting a nonprobability sample of 15,000 Japanese adults on the first wave in March 2022 and send consecutive surveys to the same respondents in September 2022 and February 2023. In each wave, we implement a fully randomized conjoint experiment that shows 2 hypothetical allocations (indicated by the percentage of increased consumption tax) to 5 dimensions of public policy: 1) public works, 2) education, 3) public health care, 4) the public pension fund, and 5) poverty relief. Any residual is assumed to be spent for government debt redemption. Respondents choose between the 2 hypothetical policy packages in a task. We assign 5 tasks to each respondent at each wave.
4. Treatment
In the first wave of March 2022, we provide the randomly selected respondents with statistical information on the relative poverty rate in Japan or a narrative on the relative poverty in Japan. Statistical evidence is cited from the National Livelihood Survey 2019 conducted by the Ministry of Health, Labour and Welfare, the government of Japan (https://www.mhlw.go.jp/toukei/saikin/hw/k-tyosa/k-tyosa19/index.html).
5. Results
5.1 Persistency of intervention effects
We investigate how long the effects of information treatment on preferences for public policies persist.
5.2 Levels and lengths of different modes of information
We investigate whether either narrative evidence or statistical evidence for relative poverty in Japan affects policy preferences more strongly than the other and whether either of the effects persists longer than the other.
Intervention
1. Information treatment
In the first wave planned in March 2022, we provide the randomly selected respondents with statistical information about the relative poverty rate in Japan or a narrative on the relative poverty in Japan. Statistical evidence is cited from the National Livelihood Survey 2019 conducted by the Ministry of Health, Labour and Welfare, the government of Japan (https://www.mhlw.go.jp/toukei/saikin/hw/k-tyosa/k-tyosa19/index.html).
2. Choice of hypothetical policy packages
We adopt a fully randomized conjoint design to generate 2 5-dimensional public policy package options. Since each level (i.e., budget allocation percentage) of an attribute (i.e., a policy) is selected in forming a policy package, we can identify a change in preference accompanying a change in attribute levels as a causal effect from the latter to the former.
Experimental design
1. Survey design
1. Panel experiment of information treatment effects
We conduct a panel Internet survey by recruiting a nonprobability sample of 15,000 Japanese adults on the first wave in March 2022 and send the consecutive surveys to the same respondent in September 2022 and February 2023. On each wave, we implement a fully randomized conjoint experiment that shows 2 hypothetical allocations indicated by the percentage of raised consumption tax to 5 dimensions of public policies; 1) public works; 2) education; 3) public healthcare; 4) public pension; 5) poverty relief. Any residual is assumed to be spent for government debt redemption. Respondents choose between the 2 hypothetical policy packages at a task. We assign 5 tasks to each respondent on each wave.
2. Information treatment
In the first wave planned in March 2022, we provide the randomly selected respondents with statistical information about the relative poverty rate in Japan or a narrative on the relative poverty in Japan. Statistical evidence is cited from the National Livelihood Survey 2019 conducted by the Ministry of Health, Labour and Welfare, the government of Japan (https://www.mhlw.go.jp/toukei/saikin/hw/k-tyosa/k-tyosa19/index.html).
3. Background characteristics
We survey demographic characteristics, educational backgrounds, income, household income, partisanship, self-conceived right-leaning, preference for the size of government, and self-conceived social status.
Design details
Cited by
No citation data available for this registration yet.