AI in the Newsroom: Experimental Evidence on the Effectiveness of AI-Generated Journalism

How to cite

Glogowsky, Ulrich; Fahn, Matthias; Brottrager, Michael; Fux, Irina. 2025. "AI in the Newsroom: Experimental Evidence on the Effectiveness of AI-Generated Journalism." AEA RCT Registry. AEARCTR-0016594. https://doi.org/10.1257/rct.16594-1.2

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ulrich Glogowsky* , Matthias Fahn , Michael Brottrager , Irina Fux

* Primary investigator

Registry ID
AEARCTR-0016594
DOI
https://doi.org/10.1257/rct.16594-1.2
First registered
Last updated
Status
In development
Trial dates
2025-09-03 – 2027-08-17
Countries
Private
JEL codes
C93, O33, L82, D83
Registry record
http://www.socialscienceregistry.org/trials/16594

Abstract

Generative AI is rapidly entering domains traditionally reserved for human expertise, including the creation of journalistic content. This development raises fundamental questions about its potential role in journalism. Our project provides field-experimental evidence from a real newsroom setting by partnering with a prominent regional newspaper. Readers are randomly assigned to news articles that are (a) written by journalists, (b) produced by large language models (LLMs), or (c) created through human-AI collaboration. All articles are based on identical scientific sources. We measure effects on article clicks, post-click engagement, and other related outcomes. Hereby, we offer causal insights into how human, AI, and hybrid authorship shape public attention to news.

Intervention

Readers are randomly assigned to news articles that are (a) written by journalists, (b) produced by large language models (LLMs), or (c) created through human-AI collaboration. All articles are based on identical scientific sources.

Experimental design

Readers are randomly assigned to news articles that are (a) written by journalists, (b) produced by large language models (LLMs), or (c) created through human-AI collaboration. All articles are based on identical scientific sources and follow a standardized editorial process. We measure effects on article clicks, post-click engagement, and other related outcomes.

Design details

Randomization method
Stratified randomization by a computer. See PAP.pdf for details.
Randomization unit
Individual users. See PAP.pdf for details.
Clusters
Individual randomization. See PAP.pdf for details.
Observations
Planned Number of Observations (units): ~232,700 user-article impressions (click analysis, at least 10 articles) and ~53,540 article views (engagement analysis, after clicks, at least 10 articles). See PAP.pdf for details.
Arms
The treatment allocation shares are as follows: On-site traffic (subscribers + other on-site users): H-H: 22% H-A: 17% H-AUG: 17% A-A: 22% AUG-AUG: 22% External traffic (headline fixed to human): H-H: 33.3% H-A: 33.3% H-AUG: 33.3% The final number of observations will depend on the realized website traffic during the experiment. See PAP.pdf for details.
Primary outcomes
Article clicks Engagement time
Secondary outcomes
See PAP.pdf for details.

Cited by

No citation data available for this registration yet.

Keywords