Gender Identity, Race, Ethnicity, and Health Insurance Discrimination in Access to Mental Health Care: Evidence from an Audit Correspondence Field Experiment

How to cite

Button, Patrick; Fumarco, Luca; Harrell, Ben; Lundebjerg, Barbara; Liu, Yu. 2026. "Gender Identity, Race, Ethnicity, and Health Insurance Discrimination in Access to Mental Health Care: Evidence from an Audit Correspondence Field Experiment." AEA RCT Registry. AEARCTR-0016309. https://doi.org/10.1257/rct.16309-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Patrick Button* , Luca Fumarco , Ben Harrell , Barbara Lundebjerg , Yu Liu

* Primary investigator

Registry ID
AEARCTR-0016309
DOI
https://doi.org/10.1257/rct.16309-1.0
First registered
Last updated
Status
In development
Trial dates
2024-01-24 – 2027-12-31
Countries
United States of America (National: all states plus DC, excluding territories.)
JEL codes
I14, I11, J16, J15, C93
Registry record
http://www.socialscienceregistry.org/trials/16309

Abstract

We use an audit correspondence experiment to test if mental health providers (MHPs) (those who do talk therapy, such as therapists and psychologists) discriminate against prospective clients based on gender identity, gender, race, ethnicity, and health insurance or payment type. We send emails or messages requesting appointments to MHPs in the United States. The messages come from fictitious prospective clients who have on-average identical emails, but signal a different gender identity, gender, race, ethnicity, and health insurance status or payment method. We test for differential treatment by MHPs by comparing response quality and response rates. We will then test for statistical discrimination and other explanatory factors by determining how discrimination varies by factors such as local demographics and social attitudes, pro- or anti-trans laws, and MHP characteristics.

Note: While this pre-analysis plan was filed after we started collecting data, we filed this pre-analysis plan before doing data cleaning or data analysis (which will be starting in late January 2026). Prior to this, we only inspected incoming data to ensure there were no issues (i.e., spot checked incoming emails), but did not clean and extract data for analysis.

Intervention

To briefly summarize our overall approach of constructing prospective clients and their appointment inquiries, before getting into the specifics below under "experimental design", each mental health practitioner (MHP), the subjects in our experiment, receives either one appointment request direct message (direct message sample, or webform sample) through a common "Find a Therapist" platform, or receives two emails to their practice's listed email address (email sample).

In all these appointment request emails, our prospective clients introduce themselves briefly, mentions that they are seeking care for anxiety or depression (randomly assigned), and asks if an appointment is available. Half of the time, the prospective client is transgender or nonbinary, which we disclose in one of two ways, and the other half of the time, the prospective client is (presumed) cisgender. Independently, we randomized the race and ethnicity of the prospective client with probability 40% for White, 30% for Black, and 30% for Hispanic.

Experimental design

***Sample Inclusion/Exclusion Criteria***

We use a popular online therapist search database to collect our sample of auditable mental health care providers (MHPs). All providers listed on the database have a mental health practice license (e.g., LMFT, LC) or have clear designation that they are licensure-track, i.e., a student getting their contact hours under the supervision of a licensed provider. In order to be included in our sample, an MHP: (1) must not specialize exclusively on specific types of clients who are outside of the scope of our experiment (e.g., children, adolescents, or couples therapy), (2) must not be specialized in a type of therapy (e.g., grief, domestic violence) that would not deal with the common mental health conditions that we signal: anxiety or depression, (3) must list an individual's profile (e.g., it cannot be the profile of a clinic), (4) must provide an email option through a web form, (5) must be accepting clients (i.e., we do not contact MHPs that indicate that they are not currently accepting clients), and (6) must clearly show on their profile that they are not managed by a third party or a large mental health conglomerate like Rula, Grow Therapy, etc.

If a mental health care provider meets the inclusion criteria for this experiment, we put them into one of two samples. If they provide an email address, or we can find one from their provided, public, website, then we put the MHP into the “direct email” sample, and we send them two emails, one each from two prospective clients, as detailed below. If the MHP does not have a direct email that we can find, we put them into the “message” sample, and we send them a message through the “Email Me” webform on the therapist search platform.

***Constructing Prospective Patients and their Emails***

Please see the uploaded pre-analysis plan.

Design details

Randomization method
Computer randomization
Randomization unit
Prospective client pairs. We generate two prospective clients following the methodology detailed above. During this randomization process, we assign a starting ZIP code for where each prospective client pair searches for mental health practitioners (MHP). We do not directly randomize MHPs but rather the prospective client they get messages from is randomized, per above.
Clusters
MHPs can be considered clusters given that each gets a message from at least one (direct message sample), but often two (email sample), prospective clients. Our power analysis, explained below, was calculated based on number of emails/messages and not MHPs (clusters). Therefore, we do not have a planned number of clusters, but rather a planned overall sample size (number of messages). The number of clusters (MHPs) we have is a function of (1) which proportion of MHPs that meet our inclusion criteria have email addresses (these get two emails) or do not (these get one email, through a webform) and (2) the total amount of MHPs we are able to screen for inclusion, which is a function of our RA budget and RA productivity.
Observations
We aim to have a minimum of 11,496 "independent equivalent" emails in order to detect meaningful differences in positive response rates between (a) transgender and nonbinary (TNB) prospective clients, versus presumed cisgender prospective clients, (b) White prospective clients versus Hispanic prospective clients, and (c) White prospective clients versus Black prospective clients. If research assistant resources allow us to go beyond 11,496, then we will conduct our analysis with our full sample, but we will also estimate our main results with the first 11,496 "independent equivalent" messages. This addresses potential concerns of stopping data collection once "desired" results are achieved - a so-called "stopping rule". By "independent equivalent", we mean adjusted for the fact that, for MHPs in our direct email sample, the two emails we send them are not entirely independent. We correct for this intra-correlation between clusters (ICC) to convert the messages from the direct email sample to the equivalent messages send independently. This involves deflating the sample size number for the direct email sample by an estimate of the ICC. As is typical, we use a median ICC value of 0.2 (Lahey and Beasley, 2018). This means that one email from our direct email sample (where we send two messages) is equivalent to 0.83 messages in our webform sample. So, our number of independent equivalent messages is given by 0.83 times the number of emails sent in the direct email sample (two per MHP in this sample), plus the number of emails sent in the webform sample (one per MHP in this sample). This can also be expressed as 1.66 x MHPs in Direct Email Sample + MHPs in Webform Sample. We estimated that we needed a sample of 7,916 "independent equivalent" messages for sufficient power to detect differences in positive response rates of at least four percentage points between cisgender and TNB prospective clients, each 50% of the sample. For White versus Black or White versus Hispanic, we need 11,496 messages, given that the sample is 40% White, 30% Black, and 30% Hispanic. For Medicaid (23%) versus private insurance (23%), this is 17,209 messages, although we expect the difference in positive response rates between Medicaid and private insurance to be much larger than four percentage points. For example, to detect at least a six percentage point difference, the required sample size would be 7,740 messages. We calculated the above using G*Power (Faul et al., 2007) as follows. First, we use, from our pilot study (Fumarco et al., 2024), the positive response rate for the main non-minority group, cisgender Whites, which was 61.5%. We then determined the number of (independent) observations of TNB and cisgender prospective clients that would be required to detect a four-percentage point difference (61.5% versus 57.5%) using a two-tailed Fisher’s exact test, with Type 1 error rate (α) of 0.05 and power (1-β) of 0.95. This was 3,958 for each group. The calculation is similar for the other comparisons. All these estimated sample sizes underestimate our statistical power in practice since nearly all of our analysis will use regression analysis, which allows us to control for other factors to increase precision. References: Faul, Franz, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. “G*Power 3: A Flexible Statistical Power Analysis Program for the Social, Behavioral, and Biomedical Sciences.” Behavior Research Methods 39 (2): 175–91. https://doi.org/10.3758/BF03193146. Lahey, Joanna N., and Ryan Beasley. 2018. “Technical Aspects of Correspondence Studies.” In Audit Studies: Behind the Scenes with Theory, Method, and Nuance, edited by S. Michael Gaddis, 81–101. New York: Springer.
Arms
The experimental design summary above describes how we assign treatment arms, such as race, ethnicity, gender, transgender or non-binary status, and payment/insurance type. The above sample size calculations provide estimates of what the total sample size required is, and from there it is possible to calculate the sample sizes required for each treatment arm group by multiplying that total by the sample proportions. This gives, for example: -11,496 messages required total, of which: ---5,748 are transgender or nonbinary (50%), and 5,748 are presumed cisgender (50%) ---4,599 are White (40%), 3,449 are Black (30%), and 3,449 are Hispanic (30%)
Primary outcomes
"Positive response rate", a binary variable (1 = positive, 0 = no response or negative response), which takes several forms: a default version and six alternative versions, that broaden or narrow what is deemed a "positive response".
Secondary outcomes
Our secondary outcome variables will include: 1. "Follow up" - Did the MHP send a follow-up reply (Y/N)? 2. Length of response (number of words). (Non-response coded as 0 words) In addition, we will analyze how time to response varies by re-estimating the default primary outcome variable but using cut-offs of 3 days, 6 days, 9 days, 12 days, 15 days, 18 days, 21 days (default), 23 days, 26 days, 29 days. From this, we can see if there is a pattern whereby certain prospective clients receive responses sooner. This approach avoids bias from estimating results just off of the sub-sample that received responses, which violates the randomization. We will analyze all these secondary outcomes for our main estimates, only. That is, we do not plan to use these for analysis such as testing factors that might affect discrimination (e.g., therapist characteristics). Another type of secondary analysis we will do is identify patterns in MHP referrals in general, and if this differs by prospective client characteristics. Appendix C provides additional details, including detailed coding of how we code for various characteristics of positive and negative referrals, beyond just those used to classify referrals as positive or negative for the purposes of coding the primary outcome variable (and its alternatives).
Sponsors
Name: Project was previously funded by NSF CAREER grant, but was terminated in April 2025 for political reasons. Name: Alfred P. Sloan Foundation Location: New York, NY, USA Url: https://sloan.org/

Cited by

No citation data available for this registration yet.

Keywords