Prosocial Ranking Challenge

How to cite

Stalinski, Mateusz; Stray, Jonathan; Beknazar-Yuzbashev, George; Budak, Ceren; Kamin, Julia; Rutherford, Kylan. 2024. "Prosocial Ranking Challenge." AEA RCT Registry. AEARCTR-0014274. https://doi.org/10.1257/rct.14274-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Mateusz Stalinski* , Jonathan Stray , George Beknazar-Yuzbashev , Ceren Budak , Julia Kamin , Kylan Rutherford

* Primary investigator

Registry ID
AEARCTR-0014274
DOI
https://doi.org/10.1257/rct.14274-1.1
First registered
Last updated
Status
In development
Trial dates
2024-07-08 – 2025-06-30
Countries
United States of America ()
JEL codes
C93, D83, L82, L86, P16, Z13
Registry record
http://www.socialscienceregistry.org/trials/14274

Abstract

More information about the study will be available after the completion of the trial.

Intervention

Experimental intervention is introduced through a browser extension that all participants have to install. This makes various modifications to which content people see on Facebook, X, and Reddit.

Experimental design

Information on the experimental design is hidden until the end of the trial.

Design details

Randomization method
The browser extension assigns the user one of the experimental groups using a random number generator. The control group will be twice as large as each treatment group.
Randomization unit
Individual
Clusters
N/A
Observations
The number of observations depends on how successful we will be in recruiting users to install the browser extension and how many of them stay in the study through the endline. We are recruiting through a variety of online research and market research companies (CloudResearch, Forthright, PureSpectrum, Cint, several others). We estimate that we will be able to recruit approximately 15,000 users at baseline, and put an upper bound on retention to endline at 80% = 12,000.
Arms
We will randomly assign individuals to treatment groups with equal probabilities, with the exception that the likelihood of being assigned a control group is twice as high as being assigned any other individual treatment group.
Primary outcomes
1. Mental Health 2. Support for partisan violence 3. Meta-perceptions on support for partisan violence 4. Affective polarization 5. Intergroup empathy 6. Political knowledge 7. Meaningful connection on platforms 8. Experiences on platforms 9. Negative Emotions 10. Social media use a. Active time spent - We measure active time following the methodology specified in Beknazar-Yuzbashev et al. (2022), summed across all three treated platforms. b. Engagement rate - Measured as: total number of engagement signals submitted, divided by number of posts seen, across all platforms. Outcomes 1-9 will be measured in all surveys (baseline, midline, endline). Some questions from outcomes 3, 4, 5, and 9 will be measured using in-feed surveys injected by the extension. Lastly, outcome 10 will be measured through usage data. Heterogeneity: We will look at heterogeneity with respect to: - Political affiliation (Democrats and Independents leaning towards Democrats vs. Independents leaning towards Republicans and Republicans), - Minority status (female respondents, ethnic minorities) - High/low political content exposure (upper quartile compared to lower three quartiles) based on the benchmark period - High/low toxic content exposure (compared to median) based on the benchmark period For outcome 10, we will additionally look at heterogeneity by social media platform.
Secondary outcomes
1. Social Trust 2. Further measures of user engagement a. Total number of posts seen b. Total number of political/civic posts seen c. Average toxicity (Jigsaw) of posts seen d. Engagement rate with toxicity - Measured as: average toxicity of posts weighted by share of total viewport time - Alternatively measured as: average toxicity of posts engaged with divided by average toxicity of all rendered posts, where engaged can be shares, clicks, reactions e. Engagement rate with political/civic posts - Political/civic posts will be classified using the classifier described in https://arxiv.org/abs/2403.13362 f. Average toxicity of posts created (Jigsaw) g. Attrition, per platform, defined as the fraction of users who had at least one session in at month 5 as compared to month 1, controlling for extension uninstallation. Heterogeneity: We will look at the same angles of heterogeneity as for the primary outcomes. For outcome 2, we will additionally look at heterogeneity with respect to social media platform.
Sponsors
Name: UC Berkeley Center for Human-compatible AI Location: Berkeley, CA Url: https://humancompatible.ai/ Name: Jigsaw LLC Location: New York City, NY Url: https://jigsaw.google.com/

Cited by

No citation data available for this registration yet.

Keywords