Policy Impacts are in the Eye of the Beholder

How to cite

bertoni, marco; Falco, Paolo; Guiso, Luigi; Jappelli, Tullio; Nisticò, Roberto. 2025. "Policy Impacts are in the Eye of the Beholder." AEA RCT Registry. AEARCTR-0015899. https://doi.org/10.1257/rct.15899-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

marco bertoni* , Paolo Falco , Luigi Guiso , Tullio Jappelli , Roberto Nisticò

* Primary investigator

Registry ID
AEARCTR-0015899
DOI
https://doi.org/10.1257/rct.15899-1.0
First registered
Last updated
Status
In development
Trial dates
2025-05-05 – 2025-11-02
Countries
Private
JEL codes
D83
Registry record
http://www.socialscienceregistry.org/trials/15899

Abstract

In many countries, political debates about important policies are highly polarized, with people often trusting information that matches their political views. In this study, we conduct a survey experiment on a representative sample of the Italian population to investigate how people change their beliefs about key reforms—early retirement, migrant regularization, employment protection, and guaranteed minimum income—after receiving scientific evidence. Specifically, we test whether people react differently depending on their political alignment with the news outlet reporting the information, and whether thinking about their political group’s views influences their own opinions. Our results will help us understand how political identity affects learning from evidence and support for public policies.

Intervention

Participants in the survey experiment see short descriptions of four structural reforms that took place in Italy in recent decades. They see the four reforms in random order: early retirement (Quota 100), migrant regularization (Decreto Flussi), employment protection (Jobs Act), or guaranteed minimum income (Reddito di Cittadinanza).

Each reform is randomly associated with an information treatment: (1) no scientific evidence, (2) scientific evidence reported by outlets without a specified political bias, (3) scientific evidence reported by a left-leaning news outlet, (4) scientific evidence reported by a right-leaning news outlet.

All information provided to participants is real and non-deceptive: the scientific findings and news articles cited are authentic, and sources are accurately presented.

Participants then answer questions about their perceptions of the policy’s effects, their voting intentions in a hypothetical referendum, and their beliefs about how co-partisans would vote.

The intervention tests belief updating, sensitivity to media alignment, and the role of conformism in shaping policy support.

Experimental design

This is a randomized survey experiment conducted on a representative sample of Italian adults. The survey experiment involves close to 5,000 participants.

Participants receive scientific evidence on the impacts of four structural reforms: early retirement (Quota 100), migrant regularization (Decreto Flussi), employment protection (Jobs Act), and guaranteed minimum income (Reddito di Cittadinanza).
Each participant is exposed to four treatments - one per policy: no evidence, neutral attribution, left-leaning outlet, right-leaning outlet. All information provided is real and non-deceptive. Each participant is exposed to all four policies and all four types of treatments, but the assignment of treatments to policies and the order in which policies are presented are randomized.

Participants are individually and randomly assigned to one of four groups, stratified to be homogeneous in terms of gender, age, and geographic area. Each group is exposed to a different order of treatments. Within each group, participants are then individually and randomly assigned to one of four possible orders in which they will view the four policies. This design results in 16 experimental cells, obtained from a combination of four treatment orders and four policy orders, and ensures variation in both policy-treatment combinations and the order in which they are presented.

After each policy description and news treatment, participants report their beliefs about policy impact (positive, neutral, negative) and their confidence levels. Then, they answer a question on their own voting intentions and one on their second-order beliefs about how voters sharing their political orientation would vote. The order in which own voting intentions and second-order beliefs are elicited is randomized across respondents. The randomization of this order is orthogonal to the randomization of the order policy and treatment.

Design details

Randomization method
Randomization is performed by a computer. Participants are individually assigned to one of the 16 experimental cells defined above. Treatment assignment is conducted at the individual level. In addition, the order in which participants report their own voting intentions and second-order beliefs is independently randomized at the individual level.
Randomization unit
Individual
Clusters
The assignment is at the individual level.
Observations
5,000 survey respondents
Arms
16 equally-sized treatment groups of roughly 312 respondents each, defined by a combination of policy order and treatment order.
Primary outcomes
Perceptions of Policy Impacts: Participants’ beliefs about the effects of the targeted policies, measured on a scale (positive, neutral, negative), along with reported confidence levels (“how sure are you” on a 0–100 scale). Voting Intentions: Participants' stated voting intentions in hypothetical referenda introducing the targeted policies (options: in favor, against, abstain). Second-Order Beliefs: Participants' beliefs about how voters sharing their political orientation would vote in the same hypothetical referenda (response scale: 0–100 voters in favor out of 100 voters).
Secondary outcomes
Media credibility perceptions: Participants' ratings (on a scale from 1 to 10) of the perceived credibility (factual accuracy) of the four news outlets presented (Il Fatto Quotidiano, Il Giornale, Il Sole 24 Ore, and La Repubblica). Perceived political orientation of media outlets: Participants' placement of the same four news outlets on a political spectrum ranging from extreme left to extreme right. Policy preferences: Participants’ agreement (on a scale from 1 to 10) with increasing the retirement age by one year and reallocating the savings toward six different public policy areas (poverty reduction, unemployment reduction, environmental protection, immigrant integration, public healthcare investment, military spending).

Cited by

No citation data available for this registration yet.

Keywords