Algorithm Aversion Revisited: Examining the Role of Task Creativity, Incentives, and Issue Salience in Educational Writing Tasks

How to cite

XUE, Lian. 2025. "Algorithm Aversion Revisited: Examining the Role of Task Creativity, Incentives, and Issue Salience in Educational Writing Tasks." AEA RCT Registry. AEARCTR-0014603. https://doi.org/10.1257/rct.14603-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Lian XUE*

* Primary investigator

Registry ID
AEARCTR-0014603
DOI
https://doi.org/10.1257/rct.14603-1.0
First registered
Last updated
Status
Completed
Trial dates
2024-10-10 – 2025-02-28
Countries
China (Wuhan)
Registry record
http://www.socialscienceregistry.org/trials/14603

Abstract

This study revisits the concept of algorithm aversion, particularly in the context of educational writing tasks. While algorithm aversion— the preference for human judgment over algorithmic advice— has been observed in decision-making fields like business and healthcare, it remains underexplored in domains requiring creativity, such as education. We examine how task type (creative vs. non-creative), incentive structures (positive vs. negative), and issue salience influence individuals' reliance on AI-generated content. Through an experimental design, participants engaged in writing tasks that varied by these factors, allowing us to assess their willingness to integrate AI suggestions. Our findings aim to uncover the drivers of algorithm aversion and provide insights for educators, policymakers, and AI developers. The results have implications for designing AI tools that can complement rather than replace human creativity, fostering more effective human-AI collaboration in educational settings.

Intervention

The intervention in this study involves exposing participants to AI-generated content during educational writing tasks. Participants will complete both creative (commentary) and non-creative (translation) tasks.

Experimental design

This study investigates the factors influencing algorithm aversion in educational writing tasks through a randomized experimental design. Participants will complete two types of writing tasks— creative and non-creative—while being exposed to different types of incentives and AI-generated suggestions. The design includes:
Task Type (Within-Subject Design): Participants engage in both creative (e.g., commentary) and non-creative (e.g., translation) writing tasks.
Incentive Structure (Between-Subject Design): Participants are randomly assigned to either a positive or negative incentive condition to evaluate how incentives influence reliance on AI suggestions.
AI Suggestion Type (Between-Subject Design): AI-generated examples vary, with participants exposed to either neutral or non-neutral suggestions to assess how suggestion nature affects acceptance.
We aim to explore how these factors interact and influence participants' willingness to rely on AI assistance in educational contexts.

Design details

Randomization method
The treatments are randomized at the session level using Weikeyan system -- a WeChat based experiment recruitment system to ensure randomization. There are 4000 subjects in the Weikeyan subjects pool; the system will divide the subjects into random N cohorts and send advertisements based on the cohorts for each treatment. The advertisement is send by WeChat notifications, which is the mostly commonly used phone App in mainland China.
Randomization unit
The unit observation is at the individual level
Clusters
The treatments are clustered at the session level; we plan to run 4-10 sessions per treatment. Each session consists of around 10 -15 subjects.
Observations
240 to 400 participants in total: 60-100 participants/treatment * 4 treatments.
Arms
We plan to recruit 60-100 participants for each treatment.
Primary outcomes
Reliance on AI Suggestions and Task Performance
Secondary outcomes
Task completion time and Confidence

Cited by

No citation data available for this registration yet.

Keywords