Trust or Verify? Automation Bias in Physician-LLM Diagnostic Reasoning

How to cite

Qazi, Ihsan Ayyub; Ali, Ayesha; Khawaja, Muhammad Asadullah; Sheikh, Ali Zafar; Akhtar, Muhammad Junaid. 2025. "Trust or Verify? Automation Bias in Physician-LLM Diagnostic Reasoning." AEA RCT Registry. AEARCTR-0015864. https://doi.org/10.1257/rct.15864-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Ihsan Ayyub Qazi* , Ayesha Ali , Muhammad Asadullah Khawaja , Ali Zafar Sheikh , Muhammad Junaid Akhtar

* Primary investigator

Registry ID
AEARCTR-0015864
DOI
https://doi.org/10.1257/rct.15864-1.0
First registered
Last updated
Status
In development
Trial dates
2025-05-20 – 2025-07-15
Countries
Private
JEL codes
I11, C91, J24, D83
Registry record
http://www.socialscienceregistry.org/trials/15864

Abstract

Diagnostic errors represent a significant cause of preventable patient harm in healthcare systems worldwide. Recent advances in Large Language Models (LLMs) have shown promise in enhancing medical decision-making processes. However, there remains a critical gap in our understanding of how automation bias---the tendency to over-rely on technological suggestions---influences medical doctors' diagnostic reasoning when incorporating these AI tools into clinical practice.

This randomized controlled trial (RCT) aims to systematically measure the extent and patterns of automation bias among medical doctors when utilizing ChatGPT-4o in clinical decision-making. We will assess how access to LLM-generated information influences diagnostic reasoning through a novel methodology that precisely quantifies automation bias. In our study design, we will randomly assign participants to one of two groups. The treatment group will receive LLM-generated recommendations containing deliberately introduced errors in a subset of cases, while the control group will receive LLM-generated recommendations without such deliberately introduced errors.

Prior to participation, all medical doctors will complete a comprehensive training program covering LLM capabilities, prompt engineering techniques, and output evaluation strategies. Responses will be evaluated by blinded reviewers using a validated assessment rubric specifically designed to detect uncritical acceptance of erroneous information, with greater score disparities indicating stronger automation bias.

Intervention

Treatment group will be provided access to LLM generated recommendations, some of which will contain deliberately flawed diagnostic information.

Experimental design

The trial will be designed as a randomized, two-arm, single-blind parallel group study.

Design details

Randomization method
Randomization done in office by a computer
Randomization unit
Individual medical doctor.
Clusters
No clustering. Individual medical doctors will be randomized.
Observations
No clustering. Individual medical doctors will be randomized.
Arms
25 medical doctors each in both treatment and control
Primary outcomes
The primary outcome will be the diagnostic reasoning score, calculated as a percentage score (0-100%) for each case.
Secondary outcomes
Final diagnosis score, calculated as a percentage score (0-100%) for each case.

Cited by

No citation data available for this registration yet.

Keywords