The Editor vs. the Algorithm: Targeting, Data and Externalities in Online News

How to cite

Sen, Ananya; Claussen, Jorg; Peukert, Christian. 2019. "The Editor vs. the Algorithm: Targeting, Data and Externalities in Online News." AEA RCT Registry. AEARCTR-0004264. https://doi.org/10.1257/rct.4264-3.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ananya Sen* , Jorg Claussen , Christian Peukert

* Primary investigator

Registry ID
AEARCTR-0004264
DOI
https://doi.org/10.1257/rct.4264-3.0
First registered
Last updated
Status
Completed
Trial dates
2017-12-01 – 2018-04-30
Countries
Germany ()
Registry record
http://www.socialscienceregistry.org/trials/4264

Abstract

We run a large randomized eld experiment with a major news outlet in Germany to study the economics
of arti cial intelligence in the case of online news. We ask whether automated recommendation can
outperform a human editor in terms of user engagement, and how this relationship depends on the amount
of personal data that is available to the algorithm. In particular, we want to assess the economic returns to data for an algorithm.
We then investigate the unintended consequences of personalized recommendation, including potential changes in the distribution of news consumption across
article categories.

Experimental design

We team up with a major German news outlet to carry out a field experiment. The landing page of the news outlet's website is curated by a (human) editor. At each point in time, N articles are featured on the landing page. Their relative placement in one of the n slots may change over the course of a news cycle, and in response to user engagement.
In general, at a given point time, any user that arrives at the landing page sees the same content in the same placement. In the experiment, every time a users visits the landing page, she is randomly assigned to a control or treatment condition. If a user is assigned to a control condition then all the articles she observes on the landing page are the ones curated by the human editor. In the treatment condition, we customize the landing page by allowing a recommendation algorithm decide which of the N articles to be placed on a specific (fixed) slot n=4. The underlying model is trained on fine grained data about the past reading behavior of each individual user, starting with data from November 2017, with the objective of maximizing engagement of any user assigned to the treatment. The initial phase of the experiment has been successfully implemented and we have experimental data for a period of five months, from December 2017 to April 2018.

Design details

Randomization method
Randomization done with a software package used by the data science team of the partner company.
Randomization unit
individual
Clusters
Anyone who visits the homepage of the news outlet online is randomly assinged to a treatment or control group. Since this is a large news outlet, we would have several million individuals treated.
Observations
Randomization takes place in a continuous way, so it depends on how many users visit the page online. We expects millions of user-sessions.
Arms
We have ended up with over 100 million user-sessions.
Primary outcomes
Clicks by the online readers of the webpage

Cited by

No citation data available for this registration yet.

Keywords