Overcoming Medical Overuse with AI Assistance: An Experimental Investigation

How to cite

Wang, Ziyi; Xue, Lian; Wei, Lijia. 2024. "Overcoming Medical Overuse with AI Assistance: An Experimental Investigation." AEA RCT Registry. AEARCTR-0014109. https://doi.org/10.1257/rct.14109-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ziyi Wang* , Lian Xue , Lijia Wei

* Primary investigator

Registry ID
AEARCTR-0014109
DOI
https://doi.org/10.1257/rct.14109-1.0
First registered
Last updated
Status
On going
Trial dates
2024-07-25 – 2024-08-31
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/14109

Abstract

This study explores the potential of Artificial Intelligence (AI) in mitigating medical overuse, specifically analyzing physicians' decision-making processes in a controlled experimental environment. Medical overuse, defined as unnecessary interventions that may cause harm to patients and increase healthcare costs, is a significant problem in both developed and developing healthcare systems. By integrating AI tools that provide evidence-based recommendations, this study aims to understand the extent to which AI can influence physician behavior and reduce overtreatment.

This experimental design consists of a field experiment conducted within a hospital, and we used a two-by-three factorial design to manipulate monetary incentives and the availability of AI assistance between different groups of physicians. We built on previous experiments by conducting a field experiment using real physicians, excluding the effects of physician self-learning as well as ordinal effects, examining AI-assisted consultations as a between-subject variable, and exploring the effects of physicians' overuse rates and clinical decision-making accuracy in familiar and unfamiliar consultation scenarios. Key findings of the study include the frequency of overtreatment, diagnostic accuracy, and adoption of AI recommendations.

The findings are expected to provide a broader contribution based on previous research, providing policymakers and healthcare administrators with more nuanced insights about the value of integrating AI technology in healthcare settings to potentially improve healthcare efficiency and patient safety.

Experimental design

We conducted a field experiment at the Central South Hospital, utilizing a two-by-three factorial design. The participants, referred to as physicians, were recruited from this institution.

Incentive Schemes (Between-Subject): The experiment varied the incentive structures for participants when making choices in a multiple-choice medicine prescription task. Three types of incentives were tested: flat, progressive, and regressive, affecting how participants were compensated for their choices.

AI Assistance (Between-Subject): The experiment was set up for participants with and without AI-assisted clinic recommendations as an incentive structure.

Control Variables: To account for individual fixed effects and other potential confounders affecting the propensity for overtreatment, we implemented pre- and post-experiment assessments. These assessments collected data on doctors’ professional abilities, algorithm literacy, trust in algorithms, awareness of algorithms, and perceptions of algorithm fairness.

Design details

Randomization method
randomization in recruitment
Randomization unit
Individual
Clusters
According to our experimental design, participants need to be randomized equally into 6 clusters.
Observations
We aim to recruit between 180 and 240 participants.
Arms
Based on our previous medical student experiment, we did a power analysis using the mean and standard deviation of the overuse rates of participants in the Regressive group before and after utilizing the AI recommendations, we conducted efficacy analyses and estimated the required sample size to be 39 in order to compare the disparity between-subject for AI-assisted diagnosis recommendations. Therefore, we decided to recruit 30-40 participants in the 6 groups of this experiment. So the experiment was set up with a total of 6 clusters, with a cluster of roughly 30-40 participants.
Primary outcomes
The key outcome variables in this experiment are the degree of physician overtreatment, and AI on accuracy rate.

Cited by

No citation data available for this registration yet.

Keywords