Converging "Truths"? Digital Knowledge Platforms, Language Skills and Informational Barriers.

How to cite

Kummer, Michael; Steffen, Sebastian. 2024. "Converging "Truths"? Digital Knowledge Platforms, Language Skills and Informational Barriers.." AEA RCT Registry. AEARCTR-0015060. https://doi.org/10.1257/rct.15060-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Michael Kummer* , Sebastian Steffen

* Primary investigator

Registry ID
AEARCTR-0015060
DOI
https://doi.org/10.1257/rct.15060-1.0
First registered
Last updated
Status
Withdrawn
Trial dates
2024-02-01 – 2027-08-01
Countries
United States of America (World)
JEL codes
D83, L86, C45, J24, D80, C63
Registry record
http://www.socialscienceregistry.org/trials/15060

Abstract

Abstract: Recent advances in generative AI and LLMs have made it possible to compare large text corpora across languages. In this paper, we leverage these tools to study language barriers and polarization on online knowledge platforms. Specifically, we study over 160,000 Wikipedia articles on 104 historic conflicts across 15 different languages. We analyze how (dis-)similarities in their various language versions evolve over time by comparing their translations and edit histories. We do so by leveraging translation capabilities of LLMs and generative AI to analyze article similarity across languages and over time. First results confirm an (expected) home bias: specific wars or battles are better covered in languages of involved parties than by uninvolved parties. Moreover, we see significant dissimilarity in contents, particularly between the winning and the losing parties. This well-documented historic bias diminishes as language and information become more interlinked. In further preliminary results on text similarity of translated and embedded articles we see evidence that “winners write history” — articles in uninvolved languages are more like those of conflict winners and less like those of conflict losers. Ongoing work highlights language barriers to the dissemination of “truth” and how translation can reduce (historical) misinformation and polarization in digital knowledge platforms.
Keywords: Open Knowledge Platforms, AI Translation, LLMs & NLP, Flow of Information, Translation Skills

Intervention

We will measure cosine distances between Wikipedia articles' embeddings ' both across articles in different languages on the same topic as well as among different versions of the same article in the same language.

Experimental design

We will identify suitable cnadidate articles that are characterized by high cosine difference between the focal article and the average article and then randomly treat a suitable fraction of these articles with a high-visibility banner that flags the uniqueness.

Design details

Randomization method
Computer
Randomization unit
wikipdata_id (topic) - language with a holdout-sample of completely untreated topics.
Clusters
250 battles and control topics.
Observations
approx. 250,000 to 500,000 -- it depends on the observation density
Arms
300 wikipedia topics controls, approx. 150 conflict topics untreated, 150 conflict topics treated. however. because each language pair is a unit of measurement, and each topic has on average 40 language versions, we will obtain 39 distance measures on average per treated article, and thus (calculating with a conservative 30) 4500 treated language-distance observations that result from the 150 treatment arms.
Primary outcomes
Cosine Similarity Differential between treated language versions of the same article.
Secondary outcomes
Editing (activity) and convergence between articles (measured as a growth in cosine similarity)

Cited by

No citation data available for this registration yet.

Keywords