Social media has increasingly become an important source of news consumption, however, limited evidence exists on its causal effect. I will conduct a natural field experiment to measure the effects of news consumption on political opinions and political behavior. The main outcomes are whether participants are persuaded by news they were randomly exposed to as a result of the intervention and whether being offered content which matches the participant’s ideology increases polarization. The results will be used to compare models explaining how individuals learn when consuming news from an outlet with an ideological slant. Specifically, I will test if consumers are affected by the slant of an outlet and whether consumers suffer from confirmation bias and are persuaded only by news matching their opinion.
Experimental design
See attached plan (registered before the trial, will be published when the trial is complete)
Design details
Randomization method
Each participant will be assigned to a control, conservative or liberal treatment, based on a randomized block design by participants’ baseline ideology. Since the sample frame arrives over time, the entire sample will not be stratified in advance but will be split gradually into groups, according to subjects' ideology.
At the beginning of the survey, respondents will be asked where they position themselves ideologically on a 7-point ideological scale from very liberal to very conservative, with an additional option of “I haven't thought about it much.” Subjects will be assigned to a treatment based on where they position themselves on the scale and when they answered the question. Each block (stratum) will be composed of three sequential subjects who chose the exact same answer among the eight options in the ideology scale survey question. The first three subjects who gave the same answer in the ideology question will be randomly split to the conservative, liberal and control groups. The next three individuals with the same ideology will be also be split into the three groups, and the process will continue with each additional respondent. In practice, since subjects arrive gradually, the first subject in each stratum will be randomly assigned to one of the three treatments, the second subject will be randomly assigned to one of the two remaining treatments and the third subject will be assigned to the final remaining treatment. If there are any technical issues in determining the strata for a particular subject, the subject will be randomly assigned to one of the three treatments.
Randomization unit
Individual
Clusters
NA
Observations
Approximately 12,000 individuals will take part in the experiment. Most of the analysis will focus on the subset of subjects who complete the follow-up survey. I predict that approximately 60% of subjects who complete the baseline survey will also complete the follow-up survey.
The precise number of participants depends on budget constraints. Since subjects are recruited using ads, and the costs of ads is somewhat volatile, the final number can be smaller or greater, and will probably in the range of 10,000-15,000
Arms
1/3 - control group
1/3 - conservative treatment
1/3 - liberal treatment
Primary outcomes
Persuasion - Were subject persuaded by news they consumed
Polarization - Did subject become more polarized when consuming news matching their opinion
Secondary outcomes
Selective Exposure - Which news pages were selected
Knowledge - Did news exposure increase knowledge