Numbers tell, but words sell: Imprecision in persuasive communication

How to cite

Thaler, Michael; Toma, Mattie; Wang, Victor. 2024. "Numbers tell, but words sell: Imprecision in persuasive communication." AEA RCT Registry. AEARCTR-0013287. https://doi.org/10.1257/rct.13287-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Michael Thaler* , Mattie Toma , Victor Wang

* Primary investigator

Registry ID
AEARCTR-0013287
DOI
https://doi.org/10.1257/rct.13287-1.0
First registered
Last updated
Status
In development
Trial dates
2024-04-06 – 2025-06-30
Countries
Private
JEL codes
C91; D83; D91
Registry record
http://www.socialscienceregistry.org/trials/13287

Abstract

Experts must often choose whether to communicate using numbers or language when sharing information with policymakers and the general public. While there are many reasons why experts would send messages using language or numbers, our study emphasizes the role of message precision: numbers represent precise estimates, whereas language is imprecise because one word can be interpreted as many numerical values. We run a large-scale experiment in which we vary message senders’ incentives to directionally persuade their audience or to convey accurate information. We analyze how senders choose between message formats, testing whether directional incentives increase their likelihood of using language to communicate as compared to using numeric estimates. To understand mechanisms, we also analyze a sample of senders who choose between more-precise numeric estimates and less-precise interval ranges to communicate. We also analyze how incentives affect how senders choose messages within each format, and study how receivers respond.

Intervention

For experimental details, see the Experimental Design (Public) section.

Experimental design

This experiment is conducted in two waves: The first wave involves "Senders" and the second wave involves "Receivers." Senders and receivers each complete 8 rounds, and are randomly matched across rounds. The structure of the experiment is as follows: First, senders learn the probability of a red ball being drawn from a box (from 0-100 percent). The "True Probability" varies within subject. Then, senders choose how to communicate this probability to a participant in the receiver wave. Finally, receivers are given a sender's message and asked to predict the probability of drawing the red ball. We describe this communication process in detail below.

Receivers' payments always depend on the accuracy of their prediction. Our first main treatment arm, which varies within subject, is whether senders' payments are:
- The same as their receiver's ("aligned"),
- Increasing in their receiver's predictions ("directional-high"), or
- Decreasing in their receiver's predictions ("directional-low").

Next, we describe what messages senders can send. Messages can come in one of three "formats":
Number messages have the format: "The chance you will draw a red ball is X percent."
Language messages have the format: "It is [word/phrase] that you will draw a red ball."
Interval messages have the format: "The chance you will draw a red ball is between Y and Y+10 percent."
Senders are either assigned, between subject, to a group in which they can choose Number and Language messages, or a group in which they can choose Number and Interval messages.

Senders in the Number-or-Language (Number-or-Interval) group make two choices. They first choose the X they want to communicate for the Number message and the word (Y) they want to communicate for the Language (Interval) message. Then, they choose whether they would rather send their Number or their Language (Interval) message. One of their two messages will always be the one used, but their preferred format will sometimes not be the one used.

Receivers face these treatments as well. They are either matched with aligned, directional-high, or directional-low senders. They are either matched with Number-or-Language senders or Number-or-Interval senders. For receivers, we also vary whether they are aware of their senders' incentives or not.

After completing the main 8 rounds, subjects map a numeric probability associated to the words and phrases available in for Language messages, which we will use to map Language to Number estimates.

Finally, we will ask subjects a set of questions, including about demographics and how they use numbers and language in everyday communication, which we may use in exploratory analyses.

We will also give subjects an attention-check question, and will drop subjects from our analyses if they do not answer this question correctly.

Design details

Randomization method
Randomization for all treatments done by computer.
Randomization unit
We randomize incentives (directional-high, directional-low, aligned) within subject. We randomize whether senders and receivers are in Number-or-Language or Number-or-Interval between subject. We randomize Probability within subject. For receivers, we randomize whether they know their sender's incentives between subject. If receivers see a random format from senders, the format they see is randomized within subject.
Clusters
Senders: 500 participants Receivers: 500 participants
Observations
Senders: 8000 message choices, 4000 message format choices Receivers: 4000 predictions after messages
Arms
Senders: 1/2 of senders choose between numbers and language, 1/2 of senders choose between numbers and intervals. 1/3 of observations involve directional-high incentives, 1/3 involve directional-low incentives, and 1/3 involve aligned incentives. Receivers: 1/2 of receivers receive messages using numbers and language, 1/2 of receivers receive messages using numbers and intervals. 1/3 of observations involve directional-high sender incentives, 1/3 involve directional-low sender incentives, and 1/3 involve aligned sender incentives. 1/2 of receivers know their sender's incentives, 1/2 of receivers do not know their sender's incentives.
Primary outcomes
H1. We hypothesize that senders are less likely to choose to send Number messages, and are more likely to choose to send Language or Interval messages, when they face directional incentives.
Secondary outcomes
H2. We hypothesize that, within a message format, senders slant Number, Language, and Interval messages in the direction of their directional incentives relative to the aligned, and we hypothesize that there is more directional slant for Language and Intervals relative to Number when senders have directional-high compared with directional-low incentives. H3. We hypothesize that senders with directional incentives who see "bad news" about the true probability use Language and Intervals more than senders who see "good news." H4. We test whether receivers are persuaded by senders; that is, we test whether their predictions move more in the direction of the sender's incentives. Then, we test whether senders with directional incentives persuade receivers more in the conditions where Language/Interval messages are sent relative to the conditions where Number messages are sent. We also consider several exploratory analyses. For senders, we will explore whether senders with directional incentives who choose the Language or Interval format slant more in the direction of their incentives compared with those who choose the Number format, and whether senders with directional incentives who see "bad news" about the true probability slant more in this direction than senders who see "good news." We may also explore other heterogeneities at the sender level. For receivers, we will explore whether receivers who do not know the incentives of the sender are persuaded more, and whether receivers who do not know the incentives of the sender are driving the persuasion effects for Language/Intervals tested in H4. We may also explore how receivers generally react to different message formats, testing whether they update their beliefs more from Number messages and rating Number messages as being more informative and honest. We may also explore other questions, such as about day-to-day usage of numbers and language, numeracy, other heterogeneities, and open-ended questions.

Cited by

No citation data available for this registration yet.

Keywords