AI, Information Processing and Dissemination Study 2
How to cite
Pethig, Florian; Bauer, Kevin; Hoehle, Hartmut. 2023. "AI, Information Processing and Dissemination Study 2." AEA RCT Registry. AEARCTR-0011541. https://doi.org/10.1257/rct.11541-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Humans increasingly engage with information created or augmented by artificial intelligence (AI). While prior work mainly considers the demand for information, little work has studied how humans process and disseminate AI-generated information once they obtain it. Specifically, our research aims to contribute to this novel research stream by studying (i) how humans process and act upon AI-generated compared to human-generated information, and (ii) how humans disseminate AI-generated information. We will perform a series of controlled, incentivized online experiments. We leverage controlled incentivized experiments to circumvent endogeneity concerns that naturally arise when it comes to being exposed to information, e.g., self-selection, informational echo chambers, and (mis)trust in specific sources.
Experimental design
We ask participants about their news consumption and knowledge of Economics-related and COVID-related topics. Next, they read two articles that either contain information about an Economics-related topic or a COVID-related topic. After participants have read each article, they answer incentivized control questions about the article's content, trust in the information, and report their perception about the article’s objectivity, quality, leaning, and reliability (we use adapted version of the questions by Bachmann et al. 2022 and self-written items). Additionally, we ask participants to rate whether the information in the article is useful to answer prediction questions related to each article (e.g., how the unemployment rate will develop) and they should summarise the information of the article for a friend (dissemination). As their final task, participants have to answer several questions on their demographics, political orientation, and preferences.
The human-generated articles are curated texts from major US news outlets, where we combine different text pieces. To transform the human-generated information, we will use the Generative Pre-trained Transformer 3 (GPT-3), a state-of-the-art natural language processing (NLP) model. The AI-generated articles are paraphrasings from the original human-generated article that the GPT-3 model produces. In a between-subject fashion we vary participants’ knowledge about the source of the information. Overall, we employ 6 treatments where we either do not reveal the source (and the set of possible sources), where we correctly reveal the source, or where we incorrectly reveal the opposite source:
T1: human-generated, correct disclosure
T2: AI-generated, correct disclosure
T3: human-generated, incorrect disclosure
T4: AI-generated, incorrect disclosure
T5: human-generated, opacity (no knowledge about possible source)
T6: AI-generated, opacity (no knowledge about possible source)
In our analyses we will compare how the labelling and participants' prior knowledge of a topic affect their perceptions about the quality, leaning, and usefulness of the information. At the end of the study, we debrief participants in treatments T3 and T4 and inform them that the article they have read was actually generated by a human / AI.
Design details
Cited by
No citation data available for this registration yet.