An Economic Experimental Study on the Impact of AI Tags on Knowledge Dissemination

How to cite

Deng, Weiguang; Qian, Fangxu. 2026. "An Economic Experimental Study on the Impact of AI Tags on Knowledge Dissemination." AEA RCT Registry. AEARCTR-0017605. https://doi.org/10.1257/rct.17605-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Weiguang Deng* , Fangxu Qian

* Primary investigator

Registry ID
AEARCTR-0017605
DOI
https://doi.org/10.1257/rct.17605-1.0
First registered
Last updated
Status
In development
Trial dates
2026-01-31 – 2026-04-30
Countries
Private
JEL codes
C93 (Field Experiments), D03 (Behavioral Economics: Underlying Principles), D82 (Asymmetric and Private Information), L86 (Information Goods and Services, Internet Content)
Registry record
http://www.socialscienceregistry.org/trials/17605

Abstract

This study investigates how different labeling regimes for AI-generated content affect user engagement and perceived credibility through a randomized controlled trial (RCT) on a major knowledge-sharing platform. Addressing market inefficiencies caused by “AI discrimination” and inconsistent standards, we evaluate the causal impact of various explicit tags (e.g., Pure AI, Human-AI Collaborative) on user behavior. We employ a stratified block randomized design with five arms across diverse content domains, strictly controlling for content quality and account characteristics. We combine objective interaction data (e.g., click-through rates, likes) with subjective credibility ratings from post-experiment questionnaires. Our analysis focuses on three dimensions: (1) the differential negative impact of tag types on perceived credibility; (2) the mediating role of perceived credibility between labeling and user engagement; and (3) heterogeneous effects across user demographics and content fields. The results aim to provide optimized labeling strategies for platforms and empirical support for digital content regulation.

Intervention

A randomized controlled experiment is conducted on the Zhihu platform, with 5 experimental groups (1 control group + 4 treatment groups) designed to manipulate AI label characteristics. The experiment covers two types of content fields (AI inquiry high-frequency fields and general fields) to explore the impact of AI labels on users' knowledge acceptance and interaction behavior.

Experimental design

A randomized controlled trial (RCT) with 5 parallel groups, using stratified random sampling to assign users to different experimental groups. The core intervention variable is the type of AI label, with content quality, account characteristics, and release time controlled as confounding variables. The experiment focuses on comparing the differences in user behavior and perceived credibility among groups.

Design details

Randomization method
Computer-generated randomization. Using Zhihu's user background data system, stratified random assignment is performed after grouping users by age and region to ensure that the user structure of each experimental group is balanced and the randomness of group assignment is guaranteed.
Randomization unit
Computer-generated randomization. Using Zhihu's user background data system, stratified random assignment is performed after grouping users by age and region to ensure that the user structure of each experimental group is balanced and the randomness of group assignment is guaranteed.
Clusters
40 content clusters (2 sub-fields × 20 hot topics per sub-field). Each content cluster is applied to all 5 experimental groups to control the impact of content characteristics on results.
Observations
1,000 individual users (200 users per experimental group). The sample size is determined based on similar studies (1,235 valid samples for human-machine difference effects) and power calculation results, ensuring sufficient statistical power to detect medium effect sizes.
Arms
Control Group (CG): 200 users, covering 40 content clusters; Treatment Group 1 (T1): 200 users, covering 40 content clusters; Treatment Group 2 (T2): 200 users, covering 40 content clusters; Treatment Group 3 (T3): 200 users, covering 40 content clusters; Treatment Group 4 (T4): 200 users, covering 40 content clusters.
Primary outcomes
User interaction behavior indicators (number of likes, comments, collections, click-through rate); User perceived credibility score of content.
Secondary outcomes
Heterogeneous response differences of users in different age groups/regions/fields.

Cited by

No citation data available for this registration yet.

Keywords