Team Production with AI
How to cite
Sun, Gregory. 2026. "Team Production with AI." AEA RCT Registry. AEARCTR-0019019. https://doi.org/10.1257/rct.19019-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This trial aims to understand how AI use affects team production. To do so, we will randomize in two steps. First, we randomize students in a summer RA program into either getting training that encourages AI use and provides tips on how to use AI tools most effectively versus getting a training that warns about all of the pitfalls of using AI. Throughout the summer, we will then have students complete tasks in pairs, where we will randomly assign pars such that teams vary in their composition of AI trainings.
Intervention
There are two layers of interventions. The first layer is at the person level, while the second layer is at the team level, based on the assignments of the first layer.
For the first layer, the 21 students in the summer program were randomized into two AI trainings. Training 1, the AI-positive training, provided tips and tricks on how to use AI tools most effectively, and tried to show students that good AI use requires good human input, but can be highly productivity-enhancing if used correctly. Training 2, the AI-negative largely just walks through examples of papers showing pitfalls of using AI.
For the second layer, on 10 days of the program, students will be assigned to work on coding tasks writing code to implement various procedures described in John List's Experimental Economics textbook. Within each day, students will be paired into teams, and team compositions are set to be roughly even between having 1) 2 AI-negative participants, 2) 2 AI-positive participants, and 3) one of each. Teams will have 1 hour to complete the code.
Experimental design
This study examines how training shapes the way people use AI tools on collaborative coding tasks. Twenty-one students in a summer program were randomly assigned to one of two trainings. The first, an AI-positive training, offered practical tips for using AI tools effectively and emphasized that good results depend on good human input while highlighting AI's potential to boost productivity. The second, an AI-negative training, walked through examples of papers illustrating the pitfalls of relying on AI. Over 10 days of the program, students were then paired into two-person teams to write code implementing procedures from John List's Experimental Economics textbook, with one hour per task. Team compositions were balanced across three types: two AI-negative participants, two AI-positive participants, or one of each, allowing us to study how individual training interacts with team composition.
To assess whether the training changed behavior, we administer a problem set of coding tasks and look for stronger signs of AI use among students in the AI-positive condition. Submitted code is evaluated on completeness, measured by how many cases it successfully handles, and on readability as judged by a third party.
Design details
Cited by
No citation data available for this registration yet.