Cultural institutions: The Effect of Physical and Digital Layouts on Visitor Experience (Part 2)

How to cite

Deshmane, Abhishek; Albéniz, Victor Martínez de; Aouad, Ali. 2022. "Cultural institutions: The Effect of Physical and Digital Layouts on Visitor Experience (Part 2)." AEA RCT Registry. AEARCTR-0009585. https://doi.org/10.1257/rct.9585-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Abhishek Deshmane* , Victor Martínez de Albéniz , Ali Aouad

* Primary investigator

Registry ID
AEARCTR-0009585
DOI
https://doi.org/10.1257/rct.9585-1.0
First registered
Last updated
Status
In development
Trial dates
2022-06-20 – 2022-07-03
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/9585

Abstract

A fundamental operational decision faced by museums and other major cultural institutions is the design of a display arrangement for their collection of items (artworks, objects, media contents), which in turn affects the visitors' interactions with the offered experiences. In this project, we collaborate with the Van Gogh Museum in Amsterdam to carry out experiments that help us causally identify potential links between layout-level decisions made by the experience providers and the visitor engagement, satisfaction levels, and learnings. The multimedia guides provided by the museum help us record visitors' movements and activities across time and space through granular event logs. Hence, the visitors navigate through both the physical layout, i.e. the arrangement of exhibits on display, and the digital layout, i.e. the tour of "Highlight" artworks recommended by the multimedia guide. We intend to analyse the interplay between these two types of content provision on the way visitors interact with the exhibits.

This study is the second part of our experimentation programme, where we test an optimized version of the Highlights tour displayed by the multimedia guide. Using a data-driven model and a simulation-based optimization algorithm, we identify a modified composition of the Highlights tour to increase the expected length of visitors' paths. Our intervention consists in removing three artworks from the Highlights tour and adding three other ones in replacement. We study the differences in the sequential choices made by the visitors across the treatment and control groups. The corresponding effect on the visitor engagement is measured, on a first level, through a Difference-in-Differences methodology; on a second level, we use a path-based Multinomial Logit model developed in previous research (paper in writing with same authors using historical observational data).

Experimental design

The Van Gogh Museum provides multimedia guides (MMT) to its visitors, a standard offering across many museums in the world. The objective of MMTs is to enable visitors to engage with the artworks on display in an informed and interactive way. As a result of this, the visitors not only navigate the physical space in the museum, but also they also browse through the digital layout of the multimedia guide. Both types of content provision affect how visitors progress within the museum and their encounters with artworks. The MMT provides the recommendation of a 'Highlights tour' that covers the masterpieces of the museum, as well as a longer, more exploratory 'Leisure tour', which includes a number of additional artworks on display. In this study, we intend to understand the effect of the selection of artworks recommended by the MMT on visitor engagement.

Our previous research in collaboration with the Van Gogh Museum yielded a stochastic choice model to describe visitors' sequential exploration of the museum collection based on the MMT clickstream data. In this study, we leverage this model to construct an "optimized" Highlights tour based on counterfactual simulations. Our algorithm determines a subset of three artworks on the Highlights tour that are replaced by three other ones with the goal of maximising the expected number of artworks viewed by visitors, subject to various practical constraints imposed by the museum. We track the effect of this change is on visitor activity, measured by usage data of the MMT guide, and on visitor satisfaction levels, tracked through post-visit online surveys.

Design details

Randomization method
Due to the practical operational constraints faced by the museum and the presence of interference between simultaneous visitors, it is not possible to randomize the visitors' assignment to the different arms (i.e., compositions of Highlights tour) using a standard A/B test. Hence, we adopt a switchback design over two weeks, comprising one treatment and one control exposure, as explained above. Each day is assigned to a condition: control (baseline MMT composition of the Highlights tour) and treatment according to a pre-determined schedule which remains unknown to the incoming visitors. Additionally, we construct a control group for our DiD analysis using the sample of visitors that exclusively utilise the Leisure tour, thereby remaining unaffected by our interventions on the Highlights tour. Our switchback schedule comprises three weekly changes to the MMT guide, chosen to balance the day-of-week seasonality effects across the two arms. Note that since nearly all visitors make one-time visits throughout our study, there is no risk of temporal interference.
Randomization unit
The outcomes are measured at the visitor session-level. Our analysis will be conducted with time-based clustered standard errors (two hours buckets and daily). We refer to the data analysis plan of Part 1 for a description of the specifications we will use in our analysis.
Clusters
There is no random assignment of the treatment conditions. Instead, we have constructed a switchback schedule over 14 days.
Observations
While the planned number of observations is contingent on the number of visitors that show up in the museum, based on the number of visitors recorded within a time period of two weeks of the study, we expect the following numbers: Visitor movement (audio guide-based): 1200 per day * 14 days = 16800 Visitor satisfaction (survey-based): 50 per day * 14 days = 700
Arms
If C denotes the days with control condition, T are the days with the treatment condition, we expect sample size Num(.) to be: Num(C) = Num(T) = 8400
Primary outcomes
We intend to track three different types of outcome variables. They are: 1. Visitor movement and clickstream activity: We track the path taken by the visitors across the museum during their visit using granular MMT logs. Based on this, we are able to build the following metrics: a) Number of stops viewed/audio clips listened to (referred to as a "hit" hereinafter), b) Number of hits per artwork, c) Proportion of the audio clips listened to/abandonment, c) Total visit time, d) Time per artwork, and e) Number of artworks skipped. Note that from these individual outcomes, we can build our aggregate primary outcome measuring the hit rates of artworks on a day, which will be the focus of our initial DiD analysis. 2. Visitor satisfaction: For this, we set up an online survey that contains questions related to visit satisfaction. As a part of this project, we instated a system in the museum to nudge visitors to go online and fill out these surveys after finishing their visit. Through this survey, we are able to track: a) Visitor overall satisfaction, b) Quality of experience with respect to the multimedia guide, and c) Willingness of the visitor to recommend the museum to their friends/colleagues/family. 3. Visitor learning: Finally, pedagogical goals are central to the operations of the museum. Our online survey also allows us to administer a quiz which the visitors can fill out towards the end of the visit. Four multiple choice questions regarding the artworks on display are asked to the visitors with the grades and correct answers made available after the conclusion of the survey and quiz.
Partners
Name: Van Gogh Museum Partner_type: none Url: https://www.vangoghmuseum.nl/en

Cited by

No citation data available for this registration yet.

Keywords