Advertising and the Cost of Search: Evidence from an Online Field Experiment

How to cite

Firullo, Cristiana; Cheyre, Cristobal; Acquisti, Alessandro. 2026. "Advertising and the Cost of Search: Evidence from an Online Field Experiment." AEA RCT Registry. AEARCTR-0019506. https://doi.org/10.1257/rct.19506-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Cristiana Firullo* , Cristobal Cheyre , Alessandro Acquisti

* Primary investigator

Registry ID
AEARCTR-0019506
DOI
https://doi.org/10.1257/rct.19506-1.0
First registered
Last updated
Status
On going
Trial dates
2025-05-13 – 2027-05-13
Countries
Private
JEL codes
M37, D83, D12, L81, C93, D18
Secondary IDs
Carnegie Mellon University’s IRB protocol STUDY2017_00000297
Registry record
http://www.socialscienceregistry.org/trials/19506

Abstract

Online advertising is usually defended on the grounds that it informs consumers about products they might want. But advertising can also pull attention away from what a person set out to do. An ad encountered in the middle of a shopping task can widen the set of options a consumer feels obliged to consider, add steps and elapsed time between the moment a purchase goal is formed and the moment it is completed, and raise the mental effort the episode requires. Whether personalized advertising
in particular helps or hinders consumers in reaching their own goals is an open empirical question, and it bears directly on how digital advertising and privacy are regulated.

This study uses browsing data from a randomized field experiment on online advertising conducted with U.S.-based internet users. Participants were randomly assigned at enrollment to one of three advertising environments for a three-month period: no display advertising (an ad-blocking browser extension); contextual, non-personalized advertising only (an anti-tracking extension that prevents behavioral targeting); or the ordinary web, with personalized, behaviorally targeted advertising. With informed consent, participants' desktop and mobile browsing was recorded, and they additionally shared their purchase histories and
answered periodic surveys about satisfaction with their purchases and with the browsing experience itself.

We use these data to measure the cost of search: the effort a consumer expends between forming a purchase goal and completing or abandoning it. The outcomes we examine are the number of distinct search queries and distinct websites visited within a shopping session; the duration of the session and of individual page visits; the number of steps taken through the purchase funnel, from product page to cart to checkout to completed order; the probability that a session beginning with a product search ends in a purchase; the elapsed time between a consumer's first search for a product and the eventual purchase of it; and self-reported satisfaction with the purchase and with the browsing experience. We compare these outcomes across the three randomly assigned conditions, and we separately test whether the personalization of advertising matters over and above exposure to advertising as such, by contrasting the targeted and contextual conditions.

The aim is to provide a measure of consumer welfare in digital environments that
counts search effort and time-to-goal alongside prices and purchase counts. If
advertising lowers search costs, consumers in the advertising conditions should
reach their goals with fewer steps and less time; if advertising instead diverts
attention, the same consumers should take longer, search more widely, and complete
fewer of the purchases they set out to make.

This registration covers a pre-specified analysis of data from a previously
conducted randomized experiment. At the time of registration, only summary
descriptive statistics and exploratory analyses on a partial sample have been
examined.

Intervention

Participants were randomly assigned, at the time of enrollment, to one of three online advertising environments, which they then experienced during ordinary, unscripted browsing for a three-month period.

In the first condition, display advertising was suppressed: participants installed a browser extension that blocks advertisements, so that ads were largely absent from the pages they visited.

In the second condition, participants received advertising, but not personalized advertising: they installed an extension that blocks third-party tracking, with the result that the ads they saw were selected on the basis of the content of the page being viewed rather than on the basis of their own browsing history, inferred interests, or demographic profile.

In the third condition, participants browsed the ordinary web and received
personalized, behaviorally targeted advertising, as any user would in the absence of the study.

The intervention is therefore a change in the advertising environment itself, applied continuously to the participant's own browsing rather than to a constructed task. Participants were not instructed to shop, to search, or to behave in any particular way, and no advertisement was created, purchased, or placed by the research team. All conditions were experienced in participants' own browsers, on both desktop and mobile, over the same three-month window.

Experimental design

The study is a randomized field experiment on the effect of online advertising on consumer search. Participants are U.S.-based internet users recruited online. At enrollment each participant installs a browser extension on desktop and, where applicable, on mobile, and is randomly assigned by the study software to one of three advertising environments, which then applies to all of their browsing for a three-month period: display advertising suppressed; contextual, non-personalized advertising only; or the ordinary web with personalized, behaviorally targeted advertising.

Participants are not assigned tasks. They browse as they otherwise would, and the extension records their browsing, the advertisements they encounter, and, periodically, their purchase history and survey responses. Because assignment is random and the observation window is identical across conditions, differences in subsequent browsing and purchasing between conditions identify the causal effect of the advertising environment.

The analysis registered here concerns the cost of search: how much effort, how many steps, and how much elapsed time separate the formation of a purchase goal from its completion, and whether these differ by advertising condition. Outcomes are measured at the level of the browsing session and of the individual purchase, with inference clustered at the participant level.

Design details

Randomization method
Randomization is performed on Qualtrics. At the moment a participant passes the screening survey and start the onboarding survey, Qualtrics randomly assigns them to one of the three conditions using a pseudo-random number generator, with equal assignment probabilities across the three conditions. No stratification or blocking was used, and no member of the research team was involved in or able to influence individual assignments. The realized numbers of participants observed in each condition are not exactly equal. That imbalance arises after assignment, from differential completion of installation and from attrition during the three-month period, and not from the assignment probabilities, which are equal by construction. Enrollment, installation completion and attrition are reported by condition, and differential attrition is tested and bounded as described in the experimental design.
Randomization unit
The individual participant. Each participant is assigned to a single advertising condition at enrollment and remains in that condition for the whole of their three-month observation window. There is only one level of randomization: no group, household or session-level assignment is used. Outcomes measured at the intent, session, page or purchase level are repeated observations on the randomized unit, and inference accounts for this by clustering standard errors at the participant level.
Clusters
1,200 individual participants. The design is not clustered: the participant is both the unit of randomization and the cluster for inference.
Observations
1,200 participants, each observed for three months of browsing. Outcomes are measured at several levels of aggregation beneath the participant. At the planned sample size we expect on the order of 300,000 browsing sessions (approximately 3 sessions per participant-day over a ninety-day window), of which on the order of 8,000 are product-search sessions containing at least one search query issued on a merchant website; several million individual page visits; and several thousand completed purchases with associated order-history records and satisfaction responses. These projections extrapolate observed per-participant rates in the sample enrolled to date and are indicative rather than guaranteed.
Arms
400 participants assigned to the ad-blocking condition (no display advertising) 400 participants assigned to the anti-tracking condition (contextual, non- personalized advertising only) 400 participants assigned to the control condition (ordinary web, personalized behaviorally targeted advertising) Assignment probabilities are equal across the three conditions. Realized arm sizes will differ from these targets because of differential completion of installation and attrition after assignment; realized counts by condition are reported with the results.
Primary outcomes
The object of this study is the cost of search: what a consumer expends, in effort and in elapsed time, between forming a purchase intent and completing or abandoning it. Our outcomes are organized around that interval. The first group asks whether advertising changes which intents are formed at all; the remaining groups condition on an intent having been formed and ask what it costs to carry it out. Intent formation. 1a. Probability that a browsing session contains at least one purchase intent event. 1b. Number of distinct purchase intents formed per participant-day. 1c. Share of intents that are ad-originated, meaning the intent event is preceded within the same session by a click on an advertisement leading to the merchant on which the intent is recorded. 1d. Share of intents that are resumptions of an intent formed in an earlier session, as against newly initiated intents. Search breadth, conditional on intent. 2a. Number of distinct search queries issued between the intent event and its resolution. 2b. Number of distinct merchant sites at which a product page is viewed over the same interval. 2c. Number of distinct websites of any kind visited over the same interval. Search duration, conditional on intent. 3a. Active browsing time, in minutes, between the intent event and its resolution. 3b. Time spent per page visit, separately for merchant and non-merchant pages. 3c. Session duration for sessions containing an intent event. Depth of the purchase funnel, conditional on intent. 4a. Number of stages reached in the funnel, from product page to cart to checkout to completed order. 4b. Number of distinct merchants at which the cart stage is reached. Intent resolution. 5a. Probability that an intent ends in a completed purchase. 5b. Probability that an intent reaching the cart stage ends in a completed purchase, the complement of cart abandonment. 5c. Probability that an intent is abandoned without resolution by the end of the observation window. Time to goal. 6a. Elapsed calendar time, in hours, between the intent event and the completed purchase that resolves it. 6b. Number of distinct browsing sessions spanned by that interval. 6c. Number of days on which activity related to the intent is observed. Realized satisfaction. 7a. Self-reported satisfaction with the purchased product's quality, price and brand, and with the browsing experience through which it was bought, each on a seven-point Likert scale. Each outcome is compared across the three randomly assigned conditions. We test separately the effect of advertising exposure, by contrasting each advertising condition with the ad-blocking condition, and the effect of personalization, by contrasting the targeted with the contextual condition.
Secondary outcomes
Purchase volume and value, measured from order history at the participant-week level: number of items ordered, number of orders, total order value, and the logarithm of item price. Cancellations and returns as a share of items ordered. Composition of purchases: the share of orders placed with merchants the participant had not previously visited, and the share placed with the merchant at which the intent originated, which measures whether search that begins in one place ends in another. Session frequency, as the number of browsing sessions initiated per participant-day. Allocation of attention across site categories, as the share of page visits and of time accounted for by merchant sites, search engines, social media, publishers, entertainment and other categories. Ad-click behavior, as the number of page visits arriving through an advertising click and the share of merchant visits so originated. Heterogeneity of the primary effects by age, household income, education, self-assessed IT skill and stated privacy concern, all recorded at enrollment. Pre-post comparison of purchase behavior using the three months of order history preceding enrollment as a within-participant baseline.

Cited by

No citation data available for this registration yet.

Keywords