The impact of fake customer reviews

How to cite

Akesson, Jesper; Metcalfe, Robert; Monti-Nussbaum, Manuel; Hahn, Robert. 2021. "The impact of fake customer reviews." AEA RCT Registry. AEARCTR-0008343. https://doi.org/10.1257/rct.8343-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jesper Akesson* , Robert Metcalfe , Manuel Monti-Nussbaum , Robert Hahn

* Primary investigator

Registry ID
AEARCTR-0008343
DOI
https://doi.org/10.1257/rct.8343-1.0
First registered
Last updated
Status
Completed
Trial dates
2020-02-03 – 2021-10-01
Countries
United Kingdom of Great Britain and Northern Ireland ()
JEL codes
M30, D18, C90
Registry record
http://www.socialscienceregistry.org/trials/8343

Abstract

Although fake online customer reviews have become prevalent on platforms such as TripAdvisor, eBay, and Amazon, little is known about how these reviews influence consumer behavior. This paper provides the first causal estimates of the effects of fake reviews on individual consumption choices. We conduct an incentive compatible online experiment with a nationally representative sample of UK respondents (n = 10,000). Participants are asked to browse a platform resembling Amazon and to choose a product. We randomly allocate different participants to variants of the platform with different types of fake reviews. Our analysis of the experimental data yields three key findings. First, fake reviews make consumers more likely to choose poor-quality products. Second, the effect of fake reviews is smaller for those who do not trust customer reviews. Third, we show that educational interventions can help reduce the effects of fake reviews.

Intervention

Study participants are asked to complete a shopping task on a platform resembling Amazon, and there is a chance that they will receive the product that they choose (i.e., we ship it to their home). All participants are shown a ‘search page’ that displays five similar products. While all five products have the same sale price, one has been classified as a Don’t Buy product by the consumer protection organization Which?, one has been classified as a Best Buy product by Which?, and the remaining three have received mediocre product ratings.

We randomize participants into one of six experimental groups. Those in Group 1 (the control group) are only shown informative reviews (i.e., reviews where the assessments are positively correlated with the quality of the product). Those in Group 2 are shown the same reviews as those in Group 1, but the Don’t Buy product has inflated star ratings, distributed in a way that is typical for products with fake reviews (i.e., mostly 5- and 1-star ratings). Group 3 is shown the same information as Group 2, but with the addition of fake and overly positive written reviews on the product page of the Don’t Buy product. Group 4 is shown the same as Group 3, with the difference being that the fake written reviews are more easily identifiable as being fake. Group 5 is shown the same information as Group 3, with the addition of a platform endorsement for the Don’t Buy product. Group 6 is exposed to the same information as those in Group 5, but with the addition of an educational intervention, which warns participants that some customer reviews are false, and provides them with some tips regarding how to spot fake reviews. The intervention appears at the top of the product search page and does not target a particular product.

Experimental design

The experiment was conducted in February 2020, and took place within an online survey (coded using Qualtrics). We recruited a nationally representative sample of 10,000 UK adults to take part in the experiment. Participants began by responding to questions about their socio-demographic characteristics and online shopping behavior, and then completed a shopping task in an environment resembling the Amazon platform.

We randomize participants into one of six experimental groups. Those in Group 1 (the control group) are only shown informative reviews (i.e., reviews where the assessments are positively correlated with the quality of the product). Those in Group 2 are shown the same reviews as those in Group 1, but the Don’t Buy product has inflated star ratings, distributed in a way that is typical for products with fake reviews (i.e., mostly 5- and 1-star ratings). Group 3 is shown the same information as Group 2, but with the addition of fake and overly positive written reviews on the product page of the Don’t Buy product. Group 4 is shown the same as Group 3, with the difference being that the fake written reviews are more easily identifiable as being fake. Group 5 is shown the same information as Group 3, with the addition of a platform endorsement for the Don’t Buy product. Group 6 is exposed to the same information as those in Group 5, but with the addition of an educational intervention, which warns participants that some customer reviews are false, and provides them with some tips regarding how to spot fake reviews. The intervention appears at the top of the product search page and does not target a particular product.

On average, the survey took 11 minutes to complete, and participants were paid £2 in exchange for their participation. The sample was recruited via the panel provider Dynata, and respondents could complete the survey via desktop or mobile devices.

Design details

Randomization method
Randomization done using Qualtrics built-in program.
Randomization unit
Individual
Clusters
n/a
Observations
10,000 individuals
Arms
1648 individuals in Group 1, 1667 individuals in Group 2, 1647 individuals in Group 3, 1671 individuals in Group 4, 1693 individuals in Group 5, 1662 individuals in Group 6
Primary outcomes
Our main outcome of interest in this experiment is the product choice that participants made. More specifically, we are interested in whether they chose a Best Buy, Don’t Buy, or a mediocre product.
Secondary outcomes
In addition to our main outcome of interest, we collected socio-demographic data on participants such as gender, age, education, income, and region in the UK they live in. We also collected data on participants’ online shopping habits, whether they trust reviews on Amazon, the share of reviews on Amazon they think are fake, and whether they think it is easy to spot fake reviews on Amazon. These variables are used to verify the validity of the randomization, and also allow us to conduct heterogeneity analyses.

Cited by

No citation data available for this registration yet.

Keywords