Using Bayes Factor Functions to synthesize evidence from replications of experiments

How to cite

Livingston, Jeffrey; Kurt, Ezgi; Young, Laura. 2025. "Using Bayes Factor Functions to synthesize evidence from replications of experiments." AEA RCT Registry. AEARCTR-0015736. https://doi.org/10.1257/rct.15736-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jeffrey Livingston* , Ezgi Kurt , Laura Young

* Primary investigator

Registry ID
AEARCTR-0015736
DOI
https://doi.org/10.1257/rct.15736-1.0
First registered
Last updated
Status
On going
Trial dates
2025-04-03 – 2025-09-19
Countries
Private
JEL codes
D90, C11, C12
Registry record
http://www.socialscienceregistry.org/trials/15736

Abstract

Following replications of an experiment, we show how Bayes Factor Functions can be used to update our beliefs about a) whether an effect exists and b) the magnitude of the effect. We compare this to Butera et al. (2020), who show how to use a method introduced by Maniadis et al. (2015) to calculate Post-Study Probabilities that an effect exists, following a new replication of an experiment. We conduct several replications of the seminal Kahneman, Knetsch, and Thaler (1990) experiment as an illustrative example.

Intervention

We replicate the Kahneman, Knetsch, and Thaler (1990) experiment. The instructions are based on a replication and extension conducted by Gächter et al. (2022).

In the experiment, subjects are randomized (via random number generation by computer) into a role as a seller of a coffee mug or a buyer of a coffee mug.

Sellers are given a coffee mug. Each mug was purchased from the Bentley University Bookstore at a cost of $12.98 per mug. They are allowed to keep the mug with them. The experiment begins approximately 20 minutes later.

In the experiment, subjects first play two rounds for hypothetical payments. In these initial rounds:
Sellers are told they own a token. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to sell the token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. IF they do not sell their token at the randomly-chosen price, their payoff is the value of the token. If they do sell their token at the randomly chosen price, their payoff is the price. This is repeated twice, with new induced values and prices selected in each round.

In these initial rounds, buyers are told they can purchase a token if they wish. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to buy a token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. If they do not buy a token at the randomly chosen price, their payoff is $0. If they do buy a token at the randomly chosen price, their payoff is their value for the token minus the price. This is repeated twice, with new induced values and prices selected in each round.

Finally, the third round involves the purchase or sale of the mug. between $0.00 and $12.50 in 50 cent increments. Sellers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to sell their mug at each of these prices. The price is then randomly selected. If they said they would want to sell their mug at the randomly chosen price, they return the mug to us and are paid the price. If they said they would not want to sell their mug at the randomly chosen price, they keep their mug.

Buyers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to buy a mug at each of these prices. The price is then randomly selected. If they said they would want to buy a mug at the randomly chosen price, they are given a mug and pay us the randomly selected price. If they said they would not want to buy a mug at the randomly chosen price, they do not purchase a mug and no transaction occurs.

The prices chosen in each round and values induced in rounds 1 and 2 are drawn separately for each subject.

Thus far the experiment has been replicated six times.

We plan on repeating this endeavor, conducting another five replications.

Experimental design

We replicate the Kahneman, Knetsch, and Thaler (1990) experiment. The instructions are based on a replication and extension conducted by Gächter et al. (2022).

In the experiment, subjects are randomized (via random number generation by computer) into a role as a seller of a coffee mug or a buyer of a coffee mug.

Sellers are given a coffee mug. Each mug was purchased from the Bentley University Bookstore at a cost of $12.98 per mug. They are allowed to keep the mug with them. The experiment begins approximately 20 minutes later.

In the experiment, subjects first play two rounds for hypothetical payments. In these initial rounds:
Sellers are told they own a token. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to sell the token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. IF they do not sell their token at the randomly-chosen price, their payoff is the value of the token. If they do sell their token at the randomly chosen price, their payoff is the price. This is repeated twice, with new induced values and prices selected in each round.

In these initial rounds, buyers are told they can purchase a token if they wish. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to buy a token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. If they do not buy a token at the randomly chosen price, their payoff is $0. If they do buy a token at the randomly chosen price, their payoff is their value for the token minus the price. This is repeated twice, with new induced values and prices selected in each round.

Finally, the third round involves the purchase or sale of the mug. between $0.00 and $12.50 in 50 cent increments. Sellers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to sell their mug at each of these prices. The price is then randomly selected. If they said they would want to sell their mug at the randomly chosen price, they return the mug to us and are paid the price. If they said they would not want to sell their mug at the randomly chosen price, they keep their mug.

Buyers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to buy a mug at each of these prices. The price is then randomly selected. If they said they would want to buy a mug at the randomly chosen price, they are given a mug and pay us the randomly selected price. If they said they would not want to buy a mug at the randomly chosen price, they do not purchase a mug and no transaction occurs.

The prices chosen in each round and values induced in rounds 1 and 2 are drawn separately for each subject.

Thus far the experiment has been replicated six times.

We plan on repeating this endeavor, conducting another five replications.

Design details

Randomization method
random number generation by computer
Randomization unit
individual
Clusters
n/a
Observations
20-35 per replication
Arms
20-35 per replication
Primary outcomes
Willingness to pay (WTP) (for subjects who are not initially endowed with a coffee mug) Willingness to accept (WTA) (for subject who are initially endowed with a coffee mug)

Cited by

No citation data available for this registration yet.

Keywords