Toxicity and User Engagement: Online Study

How to cite

Stalinski, Mateusz; Beknazar-Yuzbashev, George; Jiménez-Durán, Rafael. 2024. "Toxicity and User Engagement: Online Study." AEA RCT Registry. AEARCTR-0014362. https://doi.org/10.1257/rct.14362-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Mateusz Stalinski* , George Beknazar-Yuzbashev , Rafael Jiménez-Durán

* Primary investigator

Registry ID
AEARCTR-0014362
DOI
https://doi.org/10.1257/rct.14362-1.0
First registered
Last updated
Status
In development
Trial dates
2024-09-16 – 2025-03-31
Countries
United States of America ()
JEL codes
D12, D90, I31, L82, Z13
Registry record
http://www.socialscienceregistry.org/trials/14362

Abstract

More information about the study will be available after the completion of the trial.

Intervention

Information on the intervention is hidden until the end of the trial.

Experimental design

Information on the experimental design is hidden until the end of the trial.

Design details

Randomization method
Qualtrics randomization
Randomization unit
Individual
Clusters
N/A
Observations
We plan to recruit approximately N=1,000 participants per group (survey completions), which gives a total of about 4,000 people for the whole study. To increase power, we will also include pilot observations (approximately 100 observations per group).
Arms
We will randomly assign individuals to four experimental groups (Treatment 1, Control 1, Treatment 2, Control 2) with equal probabilities.
Primary outcomes
1. User engagement We measure whether the participant clicks “View 3 comments” at the bottom of the treated post to uncover the comment section. The link is formatted and situated within the post in a way that closely resembles how such links appear on Facebook. On click, an image with comments (following Facebook formatting) is displayed. 2. Willingness to accept for transcribing 100 social media posts Participants use a slider from $2 to $30 to choose WTA with precision of $0.1. The person with the lowest compensation will be invited to transcribe 100 social media posts and paid the second smallest compensation. If there are multiple people with the smallest compensation, one of them will be chosen at random. The lower bound is selected to reflect that the minimum compensation per hour is $8 and that the task will take at least 15 minutes. Heterogeneity: First, we will look at heterogeneity with respect to: gender and minority status. We classify participants as members of a minority group if they (1) selected an answer other than “white” in the ethnicity question, (2) identify as a Hispanic, Latino, or a person of Spanish origin, or (3) selected a different sexual orientation than heterosexual. Moreover, as religiosity shapes norms of behavior, we will look at heterogeneity with respect to being religious (i.e., choosing a different answer than “no religion” when asked about religion or belief).
Secondary outcomes
1.Recall of toxicity A randomly selected subsample of participants (1/3rd) will be asked whether or not any of the posts that they were asked to transcribe were toxic. A standard definition of toxicity will be provided: “A statement is toxic if it is a rude, disrespectful, or unreasonable comment that is somewhat likely to make you leave a discussion or give up on sharing your perspective.” 2. Recall of hate speech A randomly selected subsample of participants (1/3rd) will be asked whether or not any of the posts that they were asked to transcribe contain hate speech. Meta’s definition of hate speech will be provided: “Hate speech is a direct attack against people on the basis of protected characteristics: race, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity, and serious disease.” 3. Recall of treatment posts A randomly selected subsample of participants (1/3rd) will be asked which of the four options provided was one of the posts they were asked to transcribe. The options will be similar to each other. For example, for Treatment 1 and Control 1, we will use: - What is so difficult to understand? I hate Mondays. - What did you just say? You hate Mondays? - What is so difficult to understand? I hate Blacks. - What did you just say? You hate Blacks? 4. Entertainment rating of posts We will ask participants to rate on a scale from 0 to 100 (using a slider) how entertaining the training transcription posts were. We will look at the same angles of heterogeneity as for the primary outcomes.

Cited by

No citation data available for this registration yet.

Keywords