AI Summaries and Online Search Behavior: Evidence from a Field Experiment on Google Search
How to cite
Sen, Ananya; Agarwal, Saharsh. 2026. "AI Summaries and Online Search Behavior: Evidence from a Field Experiment on Google Search." AEA RCT Registry. AEARCTR-0017393. https://doi.org/10.1257/rct.17393-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We study how AI-generated summaries on Google Search affect users’ engagement with both search results and downstream publisher content. To causally identify these effects, we run a large-scale field experiment using a custom-built browser extension that modifies what participants see on their search results page. The extension randomly assigns about 1,500 individuals across three conditions: a control group that sees the standard AI summaries, a “hide AI summaries” group, and an “AI mode” group that receives an enhanced AI-forward interface. Over a two-week period, we track on-page engagement by measuring link clicks, time spent on the search results page, and interactions with the AI elements. We also observe downstream behavior by recording activity on the websites users visit after clicking through. Our results provide new evidence on how AI summaries reshape online information consumption and the distribution of traffic across publishers.
Intervention
We randomly assign about 1,500 individuals across three conditions: (1) a control group that sees the standard AI summaries which is the status quo. (2) A “hide AI summaries” group in which our browser extension hides AI summaries for queries that are deemed eligible in Google search. (3) Finally, an “AI mode” group that receives an enhanced AI-forward interface whenever they search for a query on Google.
Experimental design
We randomly assign about 1,500 individuals across three conditions: (1) a control group that sees the standard AI summaries which is the status quo. (2) A “hide AI summaries” group in which our browser extension hides AI summaries for queries that are deemed eligible in Google search. (3) Finally, an “AI mode” group that receives an enhanced AI-forward interface whenever they search for a query on Google. Over two weeks, we track different engagement measures such as clicks on the search page, scroll depth, time spent on the search page, and downstream.
Apart from these outcome variables, we also want to focus on certain heterogeneity analyses. We will look at heterogeneity by query type, the position that the AI summary appears on, trust in AI-generated information online, and browsing behavior prior to the experiment. As additional analysis, we also want to look at how our treatment effects vary by age and gender.
Analysis across (1) and (2) will be done at the user-session level while comparisons with (3) will require more aggregation since the search experience would be different. In user-session analyses, standard errors will be clustered at the individual level.
Our sample criteria is the following:
(1) Google Chrome has to be the only browser used by the individual, (2) exclude if they use the device only for Prolific-like surveys. Using our extension, we also have criteria based on their browsing behavior: (a) Total number of days with any browsing history: minimum 10 out of 20 days. (b) Atleast 25 unique URL domains in last 20 days, (b) atleast 10 Google searches in last 20 days.
In our pilots, we have a handful of users who have an abnormally large number of searches on Google. Such users would be excluded from our analysis.
Design details
Cited by
No citation data available for this registration yet.