The Selective Tailwind effect of Artificial Intelligence
How to cite
Messinese, Danilo; Conti, Annamaria. 2025. "The Selective Tailwind effect of Artificial Intelligence." AEA RCT Registry. AEARCTR-0015201. https://doi.org/10.1257/rct.15201-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
What role does AI play in entrepreneurial decision making? We have partially explored this question by analyzing the impact of AI predictive tools on the performance of a large sample of US startups. We have exploited the random release of Google Analytics 4 (GA4) which introduced AI predictive tools especially useful for mobile app developers. Leveraging this shock in a difference-in-differences model, we found that post-GA4-release there is a boost in customer acquisition. However, the positive premium is driven by the upper tail of the treatment effect distribution, and not by marginal improvements. These effects are largest for innovative startups led by highly skilled founders. Shedding light on the mechanisms, we show that GA4 boosts the productivity of A/B testing tools. Overall, these findings suggest that AI predictive tools are useful for complementing skilled human capital in formulating new testable business hypotheses, especially relevant for the detection of breakthroughs.
To further explore our results obtained from large-scale data, we have built a platform for data analysis and plan to conduct a lab experiment on Prolific. The goal is to better understand the role of predictive AI tools for anomaly detection for the production of breakthrough innovations, as well as the interaction of these tools with entrepreneurs' human capital.
Intervention
The goal of this lab experiment is to test whether the use of AI for anomaly detection can improve the ability to identify innovation opportunities and whether these tools completment or not human capital. Our experiment encompasses two treatments. The first treatment is a nudge about the importance of anomalies detection to uncover novel ideas and business opportunities, delivered through a short video and an ad-hoc AI tool (details below). The second treatment is a nudge about how to leverage causal reasoning to produce innovations. This nudge will be delivered through a short video.
These two treatments will result in four groups of participants (2x2 matrix):
1. Group 1 will receive only the treatment about anomalies detection
2. Group 2 will receive only the treatment about causal reasoning
3. Group 3 will receive both treatments about anomalies detection & causal reasoning
4. Group 4 is the control group
Each group will access a platform developed by the research team to analyze data from a B2C software startup. This platform closely replicates Google Analytics 4 (GA4). Participants will be given a dataset simulating user behavior for a B2C food delivery app as of 2024, including variables such as user sessions, revenue, engagement, customer acquisition, and feature adoption. The dataset, in addition to data trends, contains anomalies that, if detected, could inspire innovative solutions. Notably, groups 1 and 3 will have access to a default option within the platform that highlights data anomalies as part of the treatment.
Experimental design
The goal of this study is to evaluate whether AI-based anomaly detection enhances the ability to identify innovation opportunities and to explore whether these tools complement or not human capital. Participants will be recruited via Prolific and tasked with analyzing data from a B2C startup, with the objective of proposing innovations based on their analysis.
Participants will be provided with an ad-hoc platform for data analysis developed by the research team with time-series about key performance metrics in 2024.
Design details
Cited by
No citation data available for this registration yet.