Generative AI in Telemedicine

How to cite

Yu, Yichen; Zhang, Xu. 2025. "Generative AI in Telemedicine." AEA RCT Registry. AEARCTR-0016116. https://doi.org/10.1257/rct.16116-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Yichen Yu* , Xu Zhang

* Primary investigator

Registry ID
AEARCTR-0016116
DOI
https://doi.org/10.1257/rct.16116-1.0
First registered
Last updated
Status
In development
Trial dates
2025-01-20 – 2025-09-08
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/16116

Abstract

This research project aims to study the impact of providing a generative AI tool to physician assistants who provide ongoing care to patients through a telemedicine platform. In collaboration with a telemedicine company, we will conduct a randomized controlled trial to examine the following research questions:

1) What is the impact of the AI tool on physician assistants’ productivity and quality of service, as well as on patients’ medical adherence and engagement with the platform?
2) How do these effects vary across physician assistants with different levels of experience and expertise, and how do they evolve over time?
3) How does the use of patient-specific information to personalize AI-generated output affect outcomes compared to using a generic AI model?

Intervention

The intervention involves providing physician assistants at a telemedicine company with one of two versions of a generative AI tool designed to support continuity of care. The tool generates draft responses to patients’ inquiries. Participants in the treatment groups receive one of the following:

Treatment Arm 1: A generative AI tool that produces draft responses without incorporating patient-specific information.
Treatment Arm 2: A generative AI tool that produces personalized draft responses using patient-specific information.

Experimental design

The study involves around 900 physician assistants employed at a telemedicine company that delivers continuity of care to patients by answering patient inquiries and scheduling follow-up visits.

The experiment begins with a control period, during which all physician assistants follow their usual workflow without access to the AI tool. Toward the end of this period, participants complete a baseline survey assessing their workflow, prior experience with AI tools, and attitudes toward healthcare AI.

Following the control period, physician assistants are randomized into one of three groups at the team level:
- Control Group (1/3 of teams): Continue their usual workflow without access to the AI tool.
- Treatment Group 1 (1/3 of teams): Receive access to a basic version of the generative AI tool that does not incorporate patient-specific information.
- Treatment Group 2 (1/3 of teams): Receive access to a personalized version of the AI tool that generates draft responses using patient-specific information.

At the end of the intervention, all physician assistants complete a post-treatment survey, which includes the same questions as the baseline survey, along with additional questions about their experience using the AI tool.

Design details

Randomization method
Randomization will be conducted in-office using a computer-based random assignment procedure.
Randomization unit
Physician assistant teams.
Clusters
170 teams.
Observations
900 physician assistants.
Arms
Each group will have 1/3 of the total number of clusters. Assuming that the number of clusters will be approximately 170, this means, for example: - 57 clusters (teams) in Control Group, with no access to the AI tool. - 56 clusters (teams) in Treatment Group 1, receiving a basic version of the AI tool that generates draft responses without incorporating patient-specific information. - 56 clusters (teams) in Treatment Group 2, receiving a personalized version of the AI tool that generates draft responses based on patient-specific information.
Primary outcomes
1) Physician assistant productivity (e.g., inquiry resolution time). 2) Physician assistant response quality (e.g., accuracy, informational and emotional support, and readability). 3) Patient medical adherence (e.g., booking and attending follow-up appointments, and completing prescribed medication orders). 4) Patient engagement with the telemedicine platform (e.g., patient sentiment in responses and platform attrition).
Secondary outcomes
1) Physician assistant job satisfaction, work habits, and attitudes towards healthcare AI. 2) Physician assistant effort in reviewing and editing AI-generated drafts.

Cited by

No citation data available for this registration yet.

Keywords