Inoculation against misinformation

How to cite

Cersosimo, Matias; Koutout, Kristine; Li, Zelin; Athey, Susan. 2022. "Inoculation against misinformation." AEA RCT Registry. AEARCTR-0009721. https://doi.org/10.1257/rct.9721-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Matias Cersosimo* , Kristine Koutout , Zelin Li , Susan Athey

* Primary investigator

Registry ID
AEARCTR-0009721
DOI
https://doi.org/10.1257/rct.9721-1.0
First registered
Last updated
Status
In development
Trial dates
2022-07-09 – 2022-07-31
Countries
Kenya ()
Registry record
http://www.socialscienceregistry.org/trials/9721

Abstract

Given the current technologies, misinformation spreads around the globe at a speed faster than ever before. Once misinformation starts to propagate, it is difficult to counteract. Numerous psychological experiments have shown that displacing misinformation is challenging even in controlled lab settings (Lewandowsky et al., 2012). In this project, we instead aim to ``inoculate'' people against misinformation through teaching them how to identify and avoid sharing misinformation. In collaboration with the non-profit organization First Draft News, we developed a text message based course to inoculate people against misinformation and test its effects on sharing behavior through a survey-based field experiment in Kenya.

Intervention

We develop two text message courses, then combine them into a third, that focus on addressing different reasons people share misinformation while inoculating them to protect themselves from misinformation. First, people may share misinformation as an intuitive reaction to emotional stimulation. The "Emotions'' course focuses on teaching a "Stop and Think'' strategy to counteract this intuitive reaction. In particular, the course teaches that emotions like fear, anger, and superiority are often primed by misinformation.

Second, people may share misinformation because they do not know it is misinformation. The "Tactics'' course focuses on teaching different techniques used to create misinformation and how to identify them. Specific tactics taught by the course concern misleading graphs, imposter websites, and eyewitness media.

The "Combo'' course teaches the concepts from both the Emotions course and the Tactics course. This course addresses both potential reasons that people share misinformation to test whether the combination of skill sets from the two courses is most effective at changing participant behavior. To ensure that both the length of the text messages and the number of days in the course are the same as the Emotions and Tactics courses, the content of those courses was streamlined.

In addition to the three treatment courses, we will have two control treatments. In the "No-course'' control, participants will receive no content between the pre-survey and the post-survey. Conditional on completing the post-course survey, they will receive the Combo course. The "Reminder'' control course sends a text message each day reminding participants that misinformation is harmful for five days. The goal of this control treatment is to separately identify the effect of the course content (i.e., inoculation) and the potential reminder effect of a daily text message on misinformation.

After participants sign up for the course, they will receive one message per day for five days, unless they are a part of the control group that receives the text message course after completing the post-survey. Participants in this "no-course" control receive no content for five days, then the post-survey, then the "Combo" course. Each day’s content will be less than 1200 characters. Some days have interaction questions. Participants must answer the questions in order to proceed to the next day’s content. The primary goal of the questions is to engage participants with course content, but they also serve as a check on treatment exposure.

We aim to learn whether the skills taught by the inoculation course are substitutes for an accuracy nudge. Accuracy nudges are an effective means of decreasing misinformation sharing (see Pennycook et. al, 2021). In our experiment, the accuracy nudge is randomly assigned to users when they start the pre-survey. Participants who receive the accuracy nudge are asked the question "To the best of your knowledge, how accurate is the claim in the above post?'' for each post prior to asking the participant if they would share the post, in both surveys. Participants who do not receive the accuracy nudge are asked the accuracy question for all posts after answering the sharing question for all posts, in both surveys.

Experimental design

The experiment will consist of a pre-survey, the text message course, and a post-survey.

We will recruit our sample from the English-speaking population in Kenya who are at least 18 years old using two methods. First, we will recruit participants using Facebook advertisements. We expect the distribution of participants to be similar to Facebook users' distribution in Kenya, which is younger, more educated, and more likely to be male than the Kenyan population (see Rosenzweig, 2020). Ads will appear on Facebook and Messenger, where users will see the opportunity to "Earn AIRTIME by signing up for our FREE, five-day text message course and completing short course surveys.'' Clicking on an ad will lead users to the pre-survey. Participants are told that those who complete the pre-survey, text message course, and the post-survey will receive KSH 115 in airtime, conditional on completing all attention checks (we pay KSH 55 to participants who do not correctly answer all attention checks). In the pre-survey, participants will respond to social media posts, provide demographic information, and opt-in to a "free text message course that teaches you how to protect against misinformation.'' In the post-survey, participants will only respond to social media posts.

Participants will be able to sign up for the course at the end of the pre-survey by providing their phone number. They will also choose whether to receive the text message course via SMS or Whatsapp, a messaging platform with 97% penetration in Kenya. We will recommend that users select Whatsapp and list it as the default based on the recommendation of the text message course distributor.

The post-course survey will be embedded in the text message on the last day of the course (i.e., Day 5). In addition, we will congratulate participants for completing the course the day after the last message and remind them to take the post-survey.

Design details

Randomization method
We use the randomization feature provided by Qualtrics to assign respondents to the different courses and the accuracy nudge.
Randomization unit
We randomize treatment at the respondent level.
Clusters
1
Observations
at least 3000 participants that complete the entire procedure (pre-course survey, course, and post-course survey).
Arms
An equal number of observations per treatment.
Primary outcomes
Self-reported sharing behavior with respect to social media-like posts containing misinformation.

Cited by

No citation data available for this registration yet.

Keywords