Characterizing Misreporters: Evidence from List Experiments

How to cite

Deal, Cameron. 2025. "Characterizing Misreporters: Evidence from List Experiments." AEA RCT Registry. AEARCTR-0016555. https://doi.org/10.1257/rct.16555-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Cameron Deal*

* Primary investigator

Registry ID
AEARCTR-0016555
DOI
https://doi.org/10.1257/rct.16555-1.0
First registered
Last updated
Status
In development
Trial dates
2025-09-22 – 2025-09-23
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/16555

Abstract

Many surveys ask about topics—sexual orientation, suicidal thoughts, and polarizing policies—that people conceal due to social desirability bias. Studies that use anonymous methods (e.g., list experiments) routinely uncover large gaps in reported behavior and attitudes. For example, LGBTQ identity is 60-70% higher when anonymously reported. While list experiments have been used extensively across domains to estimate the level of misreporting, they are unable to describe the misreporters. I propose a technique for doing so. I re-interpret this design as an instrumental-variables (IV) problem: the extra item acts like an instrument that identifies the otherwise-hidden group of misreporters, allowing me to describe who they are and how they differ from people who disclose freely.
With a nationally representative sample recruited on the Prolific online platform, the project will outline the theory underlying this tool and apply it to three substantive settings: (1) economic and mental-health disparities among LGBQ Americans; (2) under-reporting of suicidal ideation; and (3) whether apparent left-right divides on abortion and climate change are exaggerated by social desirability bias.

Intervention

A treatment group of approximately 1250 respondents will see lists of 5 items, including 1 sensitive item, while a control group of roughly the same size will see shortened lists that omit the sensitive items, and be directly asked the sensitive item.

Experimental design

The study will randomly select roughly half of respondents to receive the veiled vs. direct report condition.

Design details

Randomization method
Survey software (Qualtrics)
Randomization unit
Individual
Clusters
2500 individuals
Observations
2500 individuals
Arms
1250 treatment, 1250 control
Primary outcomes
This study will use the outlined method to examine the level of misreporting in three domains: 1. LGBTQ identity 2. Mental health and treatment utilization 3. Political polarization on abortion rights and climate change Additionally, the study will use the outlined methods to characterize the following covariates for direct reporters, the full complier population, and misreporters: 1. LGBTQ identity: Earnings, employment, parent's earnings, usual weekly hours, mental health (PHQ-8 and GAD-7 scores), age, sex/gender, religious affiliation, political affiliation/ideology, vote choice, and race. 2. Mental health: Mental health (PHQ-9 and GAD-7 scores), age, sex/gender, religious affiliation, political affiliation/ideology, vote choice, and race. 3. Policy polarization: Political affiliation/ideology, vote choice, age, sex/gender. Additionally, the study will compare these results to those obtained when stratifying by the above covariate values and comparing the magnitude of underreporting via regression. Finally, to account for the possibility of misreporting in both sexual orientation and mental health status, I will obtain corrected mental health estimates via a cross-randomized list experiment with suicidal ideation and sadness/hopelessness, then use these measures to establish disparities in mental health for the LGBQ population, accounting for misreporting with respect to both the outcome and the independent variable of interest.
Secondary outcomes
The study will also use the results from the mental health component to estimate the accuracy improvements of predictive algorithms based on directly reported vs. corrected covariate values and simulate the effects of interventions under both versions of targeting. Additionally, secondary analysis will examine the efficacy of Bayesian shrinkage methods and/or empirical welfare maximization that incorporate direct reports and corrected reports to more accurately estimate covariate distributions and predict misreported outcomes. Finally, we will incorporate tests for list experiment validity such as those developed by Chuang et al (2021).

Cited by

No citation data available for this registration yet.

Keywords