Real vs. Hypothetical Stimuli in Digital-Literacy Interventions

How to cite

Enríquez, José Ramón; Athey, Susan; Koutout, Kristine. 2026. "Real vs. Hypothetical Stimuli in Digital-Literacy Interventions." AEA RCT Registry. AEARCTR-0018909. https://doi.org/10.1257/rct.18909-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

José Ramón Enríquez* , Susan Athey , Kristine Koutout

* Primary investigator

Registry ID
AEARCTR-0018909
DOI
https://doi.org/10.1257/rct.18909-1.0
First registered
Last updated
Status
In development
Trial dates
2026-06-11 – 2027-07-01
Countries
Private
JEL codes
D83, L82, O33
Secondary IDs
Stanford IRB Protocol #76130.
Registry record
http://www.socialscienceregistry.org/trials/18909

Abstract

Most survey experiments on misinformation sharing use researcher-authored hypothetical stimuli rather than real social-media stimuli. If responses differ across the two types, treatment effects may not generalize. We recruit English-speaking Nigerian Facebook users and cross-randomize each into a 2x2 design: (i) stimuli arm (Real vs. Hypothetical) and (ii) digital-literacy course (an Emotions course teaching recognition of emotional manipulation vs. a Facts-based active control). Primary outcomes are binary willingness-to-share decisions on misinformation and non-misinformation stimuli, elicited pre- and post-course. The first research question tests Real vs. Hypothetical pre-course sharing differences; the second tests whether the course effect differs by stimuli arm. Findings inform the external validity of survey-based misinformation evidence and stimulus-construction choices.

Intervention

After consent and covariate collection, each participant is cross-randomized to one of four cells. The stimuli arm factor: Real (verbatim Nigerian Facebook posts) vs. Hypothetical (the same posts edited along five linguistic dimensions—length, capitalization, present-tense rate, redaction indicators, sentiment intensity—to emulate researcher-authored stimuli from prior survey experiments). The digital-literacy course factor: an Emotions course teaching recognition of fear, anger, and superiority appeals in misinformation, vs. a Facts-based active control of comparable length covering misinformation prevalence, history, and cognitive-bias findings without manipulation-detection instruction. Both courses adapt the chatbot course of Appel et al. (2026), itself adapted from Athey, Cersosimo, Koutout, and Li (2023).

Experimental design

Between-participant 2x2 factorial cross-randomization at study entry, after pre-treatment covariates. Factors: stimuli arm (Real vs. Hypothetical) and digital-literacy course (Emotions vs. Facts-based control). Assignment is stratified on a binarization of the nine-item Digital Literacy Index (Offer-Westort, Rosenzweig, and Athey, 2024), age (cutoff 25), and sex (male vs. non-male), giving eight strata. Per-cell counters within each stratum keep the four cells balanced throughout recruitment. Sharing intentions are elicited in five blocks of six stimuli each: a pre-course binary block, an immediate post-course binary block (fixed position after the course), and three post-course blocks (a third binary block, a pairwise forced-choice block, a two-stage block) presented in uniformly randomized order at the participant level.

Within each block, three misinformation and three non-misinformation stimuli appear in randomized order with one embedded attention check at a random position. Each participant sees each of the 30 source stimuli exactly once. The first research question is identified from the pre-course block; the second from the immediate post-course block.

Design details

Randomization method
Computer-generated stratified randomization in Qualtrics. After the participant submits the covariates needed to compute their stratum, embedded JavaScript reads the per-cell counters for that stratum, assigns the cell with the smallest count (ties broken uniformly at random), and increments the counter. Assignment is invisible to the participant and reproducible from the participant-by-time log.
Randomization unit
Individual participant; single level of randomization.
Clusters
N/A
Observations
450 completed surveys, each contributing 30 sharing decisions over 30 unique stimuli across five blocks.
Arms
Balanced allocation across the four cells: ~113 participants each in Real x Emotions, Real x Facts, Hypothetical x Emotions, and Hypothetical x Facts. Total N=450, stratified across eight strata (binarized Digital Literacy Index x age x sex). Budgeting ~40% consent-to-completion attrition; the Facebook campaign targets N~750.
Primary outcomes
Three participant-level outcomes, defined for both the pre-course and the immediate post-course binary sharing blocks: (1) Misinformation sharing rate: share of three misinformation stimuli the participant would share; (2) Non-misinformation sharing rate: share of three non-misinformation stimuli the participant would share; (3) Sharing discernment: non-misinformation minus misinformation sharing rate. The first research question uses the pre-course block (Hypothetical - Real difference); the second uses the immediate post-course block as a 2x2 stimuli-by-course interaction.
Secondary outcomes
Sharing decisions from the three remaining post-course blocks: a third binary block (six decisions), a pairwise forced-choice block (three M/N pairs), and a two-stage block (three forced-choice plus six binary decisions over three M/N pairs). Subgroup heterogeneity for the first research question by digital-literacy index, pre-course misinformation and non-misinformation sharing, sex, and age. Engagement and attention-check pass rates as balance diagnostics.

Cited by

No citation data available for this registration yet.

Keywords