The spread of (mis)information: A social media experiment in Pakistan

How to cite

Hirshleifer, Sarojini; Rezaee, Arman; Raza, Agha Ali; Naseem, Mustafa. 2022. "The spread of (mis)information: A social media experiment in Pakistan." AEA RCT Registry. AEARCTR-0009954. https://doi.org/10.1257/rct.9954-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Sarojini Hirshleifer* , Arman Rezaee , Agha Ali Raza , Mustafa Naseem

* Primary investigator

Registry ID
AEARCTR-0009954
DOI
https://doi.org/10.1257/rct.9954-1.0
First registered
Last updated
Status
On going
Trial dates
2020-03-20 – 2022-10-01
Countries
Private
JEL codes
L86, L82, D80, D83, O10, I10, I15
Registry record
http://www.socialscienceregistry.org/trials/9954

Abstract

This randomized experiment takes place on a social media platform in Pakistan called Baang. It measures the impact of two treatments that fully control access to misinformation relative to a more standard ex-post approach to controlling misinformation. In the treatment versions of the platform, moderation is ex-ante: all user-generated posts are reviewed by a moderator before being posted on the platform. In the remove treatment, user content that includes COVID-19 misinformation is simply never posted to the platform, while in the sunshine treatment, user content that contains COVID-19 misinformation is posted along with a rebuttal that debunks the post. The experiment focuses on misinformation regarding COVID-19 and is combined with an intervention to disseminate official information about the pandemic on the platform.

Experimental design

This experiment tested three approaches to addressing misinformation on the platform using two treatments and a control. The two treatments rely on ex-ante moderation, which means that all user-generated content on the platform was reviewed by a moderator before being made publicly available. In the removal treatment, we never posted the content identified as misinformation related to COVID-19. In the sunshine treatment, we posted all user-generated misinformation content, but we included a specific rebuttal with each piece of content that conveyed information from trusted sources such as the WHO. These rebuttals played automatically immediately after the misinformation content, and were identified as official responses from the platform.

Design details

Randomization method
The randomization took place automatically as people called into the platform. The randomization design accounted for networks of users and minimized potential spillovers. Specifically, treatment assignment depended on how a user reached the platform for the first time after the randomization began. Original users, who called in directly, were randomly assigned to one of the three conditions when they called into the platform. Referral users, who called in because they were forwarded content from the platform by another user, were assigned to the same condition as the user who forwarded them content. Regardless of how a user came to the platform initially, once a user was assigned to a condition, they remained in that condition every time they called into the platform thereafter.
Randomization unit
Since the condition assignment of an original user determines the assignment of their referral users, this study relies on cluster-level random assignment. Each of the clusters in the study includes a single original user and their referral users if any.
Clusters
1408
Observations
3698
Arms
Of our 3698 users, 1153 were in the remove treatment arm, 1258 were in the sunshine treatment arm, and 1287 were in the control arm.
Primary outcomes
The main outcomes in the experiment examine exposure and engagement at the user-level for three sources of information. We particularly focus on the impact of the treatments on the official information posts, since these posts were designed to provide high-quality information about COVID-19. We also examine two sources of user-generated information: useful posts that contain valuable information about COVID-19 and misinformation posts that contained false information about COVID-19.

Cited by

No citation data available for this registration yet.

Keywords