LLMs, Compliance, and Decision-Making in Organizations: A Field Experiment
How to cite
Chen, Yutong. 2026. "LLMs, Compliance, and Decision-Making in Organizations: A Field Experiment." AEA RCT Registry. AEARCTR-0019059. https://doi.org/10.1257/rct.19059-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This project conducts a field experiment at a large-scale shrimp farm in China to examine how LLMs affect compliance in organizations. Workers receive AI-generated harvest recommendations, and in one treatment they can also interact with a LLM chatbot to explore the rationale behind the recommendations. Using daily pond-level data, we study how workers respond when AI recommendations conflict with managerial directives, focusing on whether LLMs improves compliance or instead provides a justification for deviation. We further test whether engagement with the AI system strengthens its behavioral impact and whether forecast errors weaken trust in AI. By linking behavioral, engagement, and economic outcomes, the study provides causal evidence on how LLMs may reshape coordination, authority, and control in organizations. The findings speak to the broader question of how information frictions within organizations undermine managerial control.
Experimental design
We will conduct a cluster-randomized field experiment at the pond level in a large shrimp-farm setting. The study has two treatment arms: in both arms, workers receive access to the same AI forecasting tool and numerical harvest recommendations as managers; in the second arm, workers also receive a chatbox interface that allows them to ask why and discuss the rationale behind the AI recommendation. The randomization unit is the pond, and all analyses will be clustered accordingly. Data are observed at the pond-day level over the production season, allowing us to measure whether workers comply with managerial directives when AI recommendations conflict with those directives. The intervention is implemented through a website using a fine-tuned forecasting model that integrates weather, hydrological, pond, and historical operational data.
Design details
Cited by
No citation data available for this registration yet.