Effect of the product order and the product review on the product choice.

How to cite

Kuroda, Toshifumi; Oiso, Hajime; Ida, Takanori; Sakaguchi, Hirohide. 2023. "Effect of the product order and the product review on the product choice.." AEA RCT Registry. AEARCTR-0011020. https://doi.org/10.1257/rct.11020-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Toshifumi Kuroda* , Hajime Oiso , Takanori Ida , Hirohide Sakaguchi

* Primary investigator

Registry ID
AEARCTR-0011020
DOI
https://doi.org/10.1257/rct.11020-1.0
First registered
Last updated
Status
In development
Trial dates
2023-03-20 – 2023-03-27
Countries
Japan ()
JEL codes
M30, D18, L81, D91, C91
Registry record
http://www.socialscienceregistry.org/trials/11020

Abstract

To investigate the impact of product reviews and product orders on consumer behavior in online retail, we will recruit 5,000 participants for the experimental websites designed to simulate an online retail store. We will assign several treatments to investigate the effect of information on consumer behavior, including two types of user reviews, three types of product orders, three types of review orders, two-sided information provisions, and a warning message against the existence of incorrect or misleading reviews.

The first type of review is based on reviews that have been deleted from an online retail website, while the other type of review is based on reviews that have not been deleted. We will sort the order of products by user rating for products, by the number of user ratings for products, and by random. We will sort the order of reviews by user rating for reviews, by newest, and by random.

To determine the willingness to pay for a product, we will use an incentive-compatible questionnaire.

Additionally, we will collect information from participants regarding their experience with online retailing, their level of trust in online retailing, risk attitudes, information-seeking attitudes, participating environments, impressions of our experimental websites, and socio-demographic characteristics. We will also record click log data from our experimental websites. Using this information, we will estimate the treatment effect of information structure on consumer behavior, the effect of the information on the consumer surplus, search cost, information processing costs, and the prior for the goods to better understand the treatment effect.

Intervention

Participants will be asked to access the websites that the investigators have designed. The first step is to answer some questions about their experience with online retailing. Next, participants will be asked to choose their preferred products from three different product categories on the website that simulate an online retail store. The website will provide natural language user reviews, the number of reviews, user ratings for products, and specifications for the products. The reviews, number of reviews, and ratings will vary among participants. In addition to the provided information, we will also assign three different orders for the products listed on the product list page and three different orders for the reviews on the product detail pages. Additionally, we will randomly assign product detail pages with two-sided information provision that shows the pros and cons of the product. Furthermore, we will randomly assign notifications on the existence of fake or infrared reviews.

Experimental design

The experiments will be conducted in March 2023. We will recruit 5,000 adult participants which represent internet users in Japan. To recruit participants, we ask an internet survey firm to send an email that asks about their willingness to participate in the online survey for online shopping with the survey periods and the fact that the survey will be conducted by academic researchers. The e-mail also informs the candidate respondents that they will be awarded 250JPY equivalent points and that a small number of them will be given the product they choose in the survey or additional points. Then, the survey firm will send e-mails with the URL of experimental websites to people who accept participating in the survey. Emails containing the URL of the experimental website are sent by the firm to candidate respondents until the obtained response can reach the target number which represents Japanese Internet users by gender and age.

The website first asks the participants about their experience in online retail. Then, the website asks respondents to choose the most wanted products or none of the products are wanted for a category of products in pages that simulate an online retail store. We pick digital cameras, headphones, and smartwatches as categories of products of store pages. We pick 10 digital cameras, 30 headphones, and 30 smartwatches for each product list, respectively. The order of categories is randomized.

The simulated store pages consist of three levels: a product listing page, a product details page, and a page displaying all reviews.

On the first level page, we show a list of products in one category. In the product list, we show the product's name, the average user rating score, the number of reviews, and the picture of the product. When respondents click on the "see details of a product" button, the website shows the 2nd level page of the product. Once the respondents proceed to the 2nd level page and return to this page, the first level page adds the "None of them are wanted" button at the top of the product list. The "None of them are wanted" button leads respondents to the "confirmation page".

On the second level page, the detail of the product contains the specifications of the product and five reviews for the product with ratings for the review. The product detail page has three buttons. The first one is the "return to the list" button, which proceeds the respondents to the first level page. The second is the "choose it" button, which proceeds respondents to the confirmation pages. The third is the "None of the products are wanted" button. The fourth is the "review is helpful" button. The fifth is "see more reviews", which proceeds respondents to the third level page. The product detail pages also may show two-sided presentations by respondents' assigned treatment.

On the third level page, we show 10 reviews and a two-sided presentation. The third level page has two buttons. The first is the "return to the page before", which proceeds respondents to the 2nd level pages. The second is the "review is helpful" button.

When the respondents proceed to the confirmation page, we show the "confirm" button and the "back to the product page" button or "back to the product list page" depending on the previous page. Once respondents push the "confirm" button, the website does not allow respondents to back to previous pages.

We randomize participants on the two sources of information, three orders of products, three orders of reviews, whether the product page contains a two-sided presentation of reviews or not, and whether the product page contains a warning message against the existence of
incorrect or misleading reviews.

After the three product categories choice experiments, we ask the respondents about their impressions and their experience of the websites, along with their experience and perceptions concerning their use of online shopping and the Internet.

Design details

Randomization method
Randomizations are done in the website server using SHA1PRNG, a pseudorandom number generation (PRNG) algorithm. For randomization unit 1) described below, adaptive randomization (biased coin design) is also adopted based on Rosenberger, W. F. (2002).
Randomization unit
Four randomizations are implemented as follows: 1) Randomization to allocate participants to 36 groups for combinations of all treatments except for a warning message. 2) Randomization to allocate participants to four groups to present different amounts of money in the WTP question. 3) Randomization to allocate participants to two groups to determine whether a warning message is to be displayed or not. 4) Randomization to allocate participants to six groups to determine orders of the three product categories to be shown on the website. For each participant, the first two randomizations are performed by each product category. This means that these two randomizations are carried out three times for a single participant. On the other hand, the last two randomizations, which determine the warning message that is expected to be effective across the categories once displayed, and also determine the order of the product categories, are performed only once for a single participant.
Clusters
1
Observations
5,000
Arms
We will assign the following treatments. 1. two sources of information (deleted: not deleted(control)) 2. three orders of products (random(control):ratings of product:number of reviews) 3. three orders of review (random(control):newest:ratings of review) 4. two types of information provisions (with notification about fake or infrared review: without it) 5. two types of products review (with two-sided presentations or without them) Therefore, the number of combinations of treatments is 72 for each product category. Therefore, we randomly assign each combination of treatments to 69 or 70 respondents at random.
Primary outcomes
Changes in the information provided and the order in which it is presented can impact consumer behavior, including product choice, information-seeking behavior, and willingness to pay for products.
Secondary outcomes
We will investigate the heterogeneity of treatment effects between the experience of using EC and interest in the product, which has been shown in previous studies. In addition, using click-log data, we can estimate the search cost and information processing costs of consumers to investigate consumer behavior in counterfactual situations.

Cited by

No citation data available for this registration yet.

Keywords