The impact of human-AI collaboration modes on worker experiences

How to cite

Ren, Yuqing; Ye, Teng; Zhang, Jundong. 2024. "The impact of human-AI collaboration modes on worker experiences." AEA RCT Registry. AEARCTR-0014729. https://doi.org/10.1257/rct.14729-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Yuqing Ren* , Teng Ye , Jundong Zhang

* Primary investigator

Registry ID
AEARCTR-0014729
DOI
https://doi.org/10.1257/rct.14729-1.0
First registered
Last updated
Status
In development
Trial dates
2024-12-02 – 2024-12-20
Countries
United States of America ()
Registry record
http://www.socialscienceregistry.org/trials/14729

Abstract

In this trial, we hope to study the impact of human AI collaboration modes on worker experiences and work quality. The research context is individual workers collaborating with a generative AI agent such as ChatGPT to perform professional writing tasks. The study will be posted as a survey on online platforms such as Prolific. We will study three collaboration modes of no AI, AI first, and human first, as well as the nature of the task (e.g., writing a report to a client vs. writing an email to co-workers) and the sentiment of the task (e.g., positive vs. negative). We will observe how these factors interact with one another to affect worker experiences and work quality. We will also examine heterogeneity among workers and how workers with different backgrounds react similarly or differently to the various task scenarios.

Experimental design

It will be a 3x2x2 = 12 between-subjects design.

We will manipulate three factors: collaboration modes, nature of the writing task, and task sentiment. The three collaboration modes are performing the writing task without AI assistance, using AI to draft the writing and human revise, human drafting and AI revise. The two types of tasks are writing a report to a client and writing an email to co-workers. The two sentiments are positive and negative.

Design details

Randomization method
The survey will be built using Qualtrics and we will use the randomization built into Qualtrics.
Randomization unit
Individual worker level
Clusters
One cluster of participants recruited from Prolific
Observations
We hope to recruit 300-500 participants from Prolific.
Arms
About 30-40 participants per condition
Primary outcomes
We will measure both worker experiences and work quality. For workers experiences, we will ask workers about their experiences of working on the writing task, particularly the level of autonomy, self-efficacy, enjoyment, and work stress they experienced. For work quality, we will use computational tools to assess complexity and readability and have human experts assess the subjective measures of work quality.

Cited by

No citation data available for this registration yet.

Keywords