Numbers Tell, but Words Sell: Evidence from Research Communication
How to cite
Thaler, Michael; Toma, Mattie; Wang, Victor Yaneng. 2024. "Numbers Tell, but Words Sell: Evidence from Research Communication." AEA RCT Registry. AEARCTR-0013947. https://doi.org/10.1257/rct.13947-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Researchers must often choose whether to communicate using numbers or language when sharing information with policymakers. While there are many reasons why experts would send messages using language or numbers, our study emphasizes the role of message precision: numbers represent precise estimates, whereas language is imprecise because one word can be interpreted as many numerical values. We run an experiment in which we recruit academic researchers and vary their incentives to directionally persuade or to convey accurate information to a sample of policymakers that we recruit. We analyze how researchers choose between message formats, testing whether directional incentives increase their likelihood of using language to communicate as compared to using numeric estimates, and explore these effects on policymakers.
Intervention
For experimental details, see the Experimental Design (Public) section.
Experimental design
Recruitment:
This experiment will be conducted in two waves: The first wave involves "Senders" (recruited from a pool of academic researchers) and the second wave involves "Receivers" (recruited from a pool of policymakers in government using the Warwick Policymakers Lab).
We will recruit senders via emails to academic social-science researchers over the course of two weeks, or until we reach 150 senders, whichever comes first. If we do not reach 50 senders during this time period, we will extend the time period. At the end of the study, we will ask senders if they previously knew about our main hypothesis, and will not include senders who answer "yes" to this question in our main analysis.
We will recruit receivers via participants in the Warwick Policymakers Lab over the subsequent eight weeks, or until we reach 75 Receivers, whichever comes first.
Experiment Timing:
Senders complete multiple rounds. In each round, they learn the treatment effect size of an academic research study. Then, they choose how to communicate this effect size to a participant in the receiver wave as detailed below.
Receivers complete multiple rounds. In each round, they are asked to predict the treatment effect of one of the studies. Then, they are matched with a sender and are given the sender's message. Finally, they are asked to predict the effect size of the study, asked how likely they would be to share the information with their colleagues, and are asked whether they would like to receive an infographic about the study that they can share with their colleagues.
At the end of the study, we will ask subjects a set of questions, including about demographics, which we may use in exploratory analyses.
We will include all completed rounds in our analysis, even in cases when the participant does not finish the survey.
Incentives:
Senders and receivers are paid in Amazon gift cards.
For receivers, one of the questions is randomly selected, and they earn a bonus payment if their prediction of the effect size is sufficiently close to the actual effect size.
For senders, we randomly vary whether their payments are:
- The same as their receiver's ("aligned"), or
- Increasing in their receiver's predictions ("directional").
Next, we describe what messages senders can send. Messages can come in one of two "formats":
- Number messages have the format: "The treatment led to an increase of X percentage points."
- Language messages have the format: "The treatment led to a [word/phrase] increase."
Senders make two choices. They first choose the X they want to communicate for the Number message and the word they want to communicate for the Language message. Then, they choose whether they would rather send their Number or their Language message. One of their two messages will always be the one used, but their preferred format will sometimes not be the one used.
Receivers face these treatments as well. That is, they are either matched with senders who have aligned or directional incentives, but do not know which type of sender they face.
Design details
Cited by
No citation data available for this registration yet.