Social Desirability Bias in Survey Measures of Sexual Identity and Attitudes Toward Homosexuality: Evidence from Double List Experiments in France

How to cite

Colomb, Hugo. 2026. "Social Desirability Bias in Survey Measures of Sexual Identity and Attitudes Toward Homosexuality: Evidence from Double List Experiments in France." AEA RCT Registry. AEARCTR-0019293. https://doi.org/10.1257/rct.19293-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Hugo Colomb*

* Primary investigator

Registry ID
AEARCTR-0019293
DOI
https://doi.org/10.1257/rct.19293-1.0
First registered
Last updated
Status
On going
Trial dates
2026-07-09 – 2026-08-05
Countries
France ()
JEL codes
C93, J15
Registry record
http://www.socialscienceregistry.org/trials/19293

Abstract

Non-heterosexual identities and attitudes toward homosexuality are commonly measured using direct survey questions, which may be affected by social desirability bias. This study examines the extent of this bias in France using three double list experiments implemented within the nationally representative ELIPSS panel. The experiments measure (i) heterosexual identification, (ii) acceptance of having a gay son, and (iii) acceptance of having a lesbian daughter. Comparing prevalence estimates obtained through indirect and conventional direct questions will provide estimates of the magnitude of response bias. The study will then examine whether response bias varies across respondents’ characteristics, whether acceptance differs according to the gender of the hypothetical child, and whether respondents’ second-order beliefs accurately reflect observed levels of acceptance and are associated with response bias.

Experimental design

This study relies on list experiments, also known as the item count technique, to estimate the prevalence of sensitive characteristics while reducing social desirability bias. Respondents are randomly assigned to either a short-list condition, containing control items, or a long-list condition, which includes the same control items plus one additional sensitive/key item. Instead of asking respondents directly whether the sensitive statement applies to them, list experiments ask respondents to report the total number of statements in the list that are true for them, without indicating which specific statements they endorse. Under random assignment, the average count of true control items is expected to be identical across groups. The difference between the average number of statements reported in the long- and short-list conditions therefore identifies the prevalence of the sensitive characteristic, under the standard assumptions of the list experiment (treatment randomization, no design effect, and no liar).
To increase statistical efficiency and strengthen the internal validity of the design, the study implements double list experiments (DLEs). For each key statement, respondents are presented with two different lists of control items (Lists A and B). Half of the respondents receive List A in its short version and List B in its long version, while the other half receive List A in its long version and List B in its short version. Prevalence is estimated separately from each list pair, and the two estimates are subsequently averaged.
The study examines three key statements that are phrased so that the potentially stigmatized response corresponds to a negative answer:
• KS1: I consider myself to be heterosexual.
• KS2: If I had a lesbian daughter, I would accept her without any problem.
• KS3: If I had a gay son, I would accept him without any problem.
Applying the DLE to each key statement results in a total of six lists: Lists 1A and 1B measure heterosexual identification, Lists 2A and 2B measure acceptance of a lesbian daughter, and Lists 3A and 3B measure acceptance of a gay son.

Design details

Randomization method
First, respondents will be randomly assigned either to a direct-only condition (10%) or to the full experimental module (90%). Second, respondents assigned to the full experimental module will then be randomly allocated across six experimental paths: • Path 1: List 1A + KS1, List 1B; List 2A + KS2, List 2B; List 3A + KS3, List 3B. • Path 2: List 1B + KS1, List 1A; List 2B + KS2, List 2A; List 3B + KS3, List 3A. • Path 3: List 2A + KS2, List 2B; List 3A + KS3, List 3B; List 1A + KS1, List 1B. • Path 4: List 2B + KS2, List 2A; List 3B + KS3, List 3A; List 1B + KS1, List 1A. • Path 5: List 3A + KS3, List 3B; List 1A + KS1, List 1B; List 2A + KS2, List 2B. • Path 6: List 3B + KS3, List 3A; List 1B + KS1, List 1A; List 2B + KS2, List 2A. Third, the order of statements within each list is randomized independently for every respondent. Finally, the DQ module includes a set of questions concerning the acceptance of a gay son and another concerning the acceptance of a lesbian daughter. The order of these two sets is randomized. Randomization will be implemented automatically by Qualtrics.
Randomization unit
Individual level randomization
Clusters
Approximately 2,400 individual respondents (individual-level randomization; no clustering).
Observations
Approximately 2,400 individual respondents, based on an expected response rate of 80% among the approximately 3,000 ELIPSS panel members.
Arms
10% of respondents will be assigned to the direct-question-only condition (approximately 240 respondents). The remaining 90% will complete the double list experiments followed by the direct questions (approximately 2,160 respondents) and will be allocated equally across six experimental paths, corresponding to approximately 15% of the total sample, or 360 respondents, per path. Independently, all respondents will be randomized in equal proportions to the order of the direct-question modules: 50% will receive the gay-son module before the lesbian-daughter module and 50% will receive the reverse order (approximately 1,200 respondents per order).
Primary outcomes
For each of the three key statements—heterosexual identification, acceptance of having a gay son, and acceptance of having a lesbian daughter—the primary outcome variables are: (i) the prevalence estimates obtained from the double list experiments (DLEs); (ii) the corresponding prevalence estimates obtained from the direct questions (DQs); and (iii) the response bias, defined as the difference between the DQ and DLE prevalence estimates.
Sponsors
Name: Centre de Données Socio-Politiques (CDSP, Sciences Po–CNRS) Location: Paris, France Url: https://www.sciencespo.fr/cdsp/fr/
Partners
Name: Centre de Données Socio-Politiques (CDSP, Sciences Po–CNRS) Partner_type: none Url: https://www.sciencespo.fr/cdsp/fr/

Cited by

No citation data available for this registration yet.

Keywords