Adaptive RCT on Misinformation
How to cite
Cibik, Ceren Bengu; D'Adamo, Riccardo; Shakhina, Natalia; Hofele, Andreas. 2025. "Adaptive RCT on Misinformation." AEA RCT Registry. AEARCTR-0015486. https://doi.org/10.1257/rct.15486-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Randomised Control Trials (RCTs) are the gold standard for testing hypotheses about competing interventions. Adaptive RCTs, which utilise real-time data for decision-making, have gained popularity in industry and are now attracting attention in academia and regulation for their efficiency in experimental studies. This project aims to apply Adaptive RCTs to explore interventions that enhance social media users' ability to identify misinformation and disinformation. Given the prevalence of algorithmic bias on social media, misinformation can spread rapidly. This experiment will uniquely bring together many behavioural interventions, which have only been tested separately, including fact-checker and AI labelling, community notes, prompts, and media literacy/inoculation techniques. By reallocating resources to better-performing interventions in real-time, this Adaptive RCT will provide robust evidence to identify the most effective strategies to combat misinformation.
Intervention
We used an existing classification of interventions against online misinformation (based on 81 scientific papers) to develop intervention ideas.^1 Afterwards, we prioritised the ideas based on feasibility, relevance, and potential impact. We will test the effect of AI and fact-checkers labelling, community notes, prompts and media literacy/inoculation techniques (together with some of their combinations as descibed below) on correctly identifying correct/misinformation.
AI labelling will include a label on the social media post stating " AI has flagged this information as fake".
Fact-checkers labelling will include the label on a social media post stating "Independent fact-checkers have flagged this information fake".
Community notes will include a note under the social media post stating "Our community members added context. Members of our community have flagged this information might be fake." with an additional post-specific explanation of why it was flagged.
The inoculation quiz will consist of a process at the beginning of the experiment where participants will be given 3 different social media posts one-by-one and asked immediately if this information is true or false. Based on their answers, they will receive immediate feedback on their answers by providing post-specific tips on how to detect misinformation.
Prompts will include receiving a quick summary of what has been taught to the participants with the inoculation quiz that will be shown after 50% of the feed. (By definition, prompts will be tested only with the inoculation quiz.)
^1 Kozyreva, A., Lorenz-Spreen, P., Herzog, S. M., Ecker, U. K., Lewandowsky, S., Hertwig, R., ... & Wineburg, S. (2024). Toolbox of individual-level interventions against online misinformation. Nature Human Behaviour, 1-9.
Experimental design
We will present each participant with a set of social media posts each containing different pieces of information (news, commercial advertisements etc.). There will be fifteen posts in total and they will be presented in a randomised order. 2/3 of the posts will include correct information, whereas 1/3 will include misinformation*. Participants will be randomised** across one of the twelve trial arms involving different interventions (or combinations thereof as described above). For each post presented, the participant will be asked to assess if the post contains misinformation or not and how confident they are in their response. After participants complete their assessment of fifteen social media posts, there will be a post-survey questionnaire. It includes media consumption habits, social media attitudes and use, general conspiracy theory beliefs and demographic information (age, gender, education, ethnicity, non-native English speaker and political leaning). We will also employ 2 attention checks throughout the experiment. The experiment will conclude with debunking misinformation that participants will see during the experiment.
*Ratio 2:1 for True vs Mis/dis posts. Ratio decision took into consideration a) making sure there are enough mis/dis posts to get primary outcome data while not making the experiment too long b) not making it completely unrealistic or making it easy to guess if used 1:1 ratio.
**Details about the randomisation can be found below in the Randomisation Section and also in the Technical Annex.
Design details
Cited by
No citation data available for this registration yet.