The Effects of Social Media Comments Section Moderation on Political Attitudes and Beliefs

How to cite

Cao, Thomas; Xu, Yiqing; Yang, Leo. 2022. "The Effects of Social Media Comments Section Moderation on Political Attitudes and Beliefs." AEA RCT Registry. AEARCTR-0010337. https://doi.org/10.1257/rct.10337-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Thomas Cao* , Yiqing Xu , Leo Yang

* Primary investigator

Registry ID
AEARCTR-0010337
DOI
https://doi.org/10.1257/rct.10337-2.0
First registered
Last updated
Status
In development
Trial dates
2022-11-11 – 2023-01-31
Countries
Private
JEL codes
H12, H83, D80, D83, D91, L86
Registry record
http://www.socialscienceregistry.org/trials/10337

Abstract

This project focuses on the intersection of propaganda and censorship in China, where state media and other accounts post propaganda content on social media and censor undesirable comments under these posts. The experimental part of this project will evaluate to what extent comment moderation affects public opinion and political attitudes. We will conduct survey experiments based on real propaganda posts and comment censorship behavior exercised by state-sponsored social media accounts.

Intervention

Treatment and control arms will see the same social media posts with different comments sections, or different background knowledge before they see the posts.

Experimental design

Respondents will be recruited via Qualtrics and Lucid in China. Each respondent will see six social media posts and will be asked to answer four questions after each post: 1) to what extent they agree or disagree with the post's position; 2) how many people in society they think agree with the post's position; 3) would you report content that criticizes the post's position; and 4) how many people in society they think would report such critical content.

Design details

Randomization method
Done by the Qualtrics randomizer
Randomization unit
In the first experiment, randomization occurs on the unit of posts. In the second and third experiments, randomization occurs on the unit of respondents. Update (Dec 17, 2022): In the new experiment, randomization also occurs on the unit of respondents.
Clusters
N/A
Observations
We will recruit approximately 1,800 ~ 2,000 respondents for the three experiments in total. Each respondent will see six different Weibo posts. Update (Dec 17, 2022): Due to difficulty in recruiting a sufficient number of respondents in China by Qualtrics, we were only able to collect approximately 1,000 responses for the first three experiments. Hence, we will conduct another experiment and use Lucid to recruit another 1,000 respondents for the new experiments (see Update in the Experimental Design section).
Arms
In the first experiment, each post will be randomly assigned into one of the four versions (Treatment Arms 1~3 and Control) with equal probability, so we will have approximately 600 respondents * 6 posts/respondents * .25 = 900 posts in each arm. In the second experiment, respondents will be randomly assigned into one of the four arms, so each arm will have approximately 600 respondents * 0.25 = 150 respondents. In the third experiment, approximately 1/3 of the respondents will be assigned to Control and 2/3 of the respondents will be assigned to Treatment, so we will have approximately 600 * 1/3 = 200 respondents in Control and 600 * 2/3 = 400 respondents in Treatment. Update (Dec 17, 2022): Due to difficulty in recruiting a sufficient number of respondents in China by Qualtrics, we were only able to collect approximately 1,000 responses for the first three experiments. Hence, we will conduct another experiment and use Lucid to recruit another 1,000 respondents for the new experiments (see Update in the Experimental Design section). The new experiment will have approximately 200 respondents in each arm.
Primary outcomes
Our main experiment is the first experiment: We will examine the respondents' second-order beliefs of 1) what proportion of people in society agrees with the regime's position represented in the propaganda social media posts and 2) what proportion of people in society will report anti-regime/critical content.
Secondary outcomes
Our secondary outcomes are 1) the extent to which the respondents agree with the regime's position represented in the propaganda social media posts and 2) whether they themselves will report anti-regime/critical content. We will also conduct two additional experiments in parallel to see how we may or may not be able to mitigate the effects of the first experiment.

Cited by

No citation data available for this registration yet.

Keywords