Assessing Generative AI value in a public sector context: evidence from a field experiment
How to cite
Fitzpatrick, Trevor; Walsh, David; Kelly, Seamus; Carey, Patrick; Nugent, Ruairi; Cleary, Tom. 2024. "Assessing Generative AI value in a public sector context: evidence from a field experiment." AEA RCT Registry. AEARCTR-0013347. https://doi.org/10.1257/rct.13347-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Generative AI economic benefits in aggregate are often cited. However, the productivity or efficiency improvements within organisations is an emerging topic. Recent research focused on knowledge work suggest productivity or efficiency improvements may be large, finding significant effects for certain types of tasks, with more benefits to less experienced workers. Evaluating the value/benefits and costs of GenAI in knowledge-driven organisations within a public sector may not be as clear-cut in a for-profit context. To provide some evidence, our study will test the impact of GenAI on two knowledge-work tasks among staff in the organisation.
Intervention
The potential productivity or efficiency improvements of Generative AI within organisations is an emerging research topic, including the potential effects for knowledge workers. Recent research (Dell'Acqua et al., 2023) focused on knowledge work suggest productivity or efficiency improvements may be large, finding significant effects for certain types of tasks, with more benefits to less experienced staff. To provide initial evidence, our study will test the impact of GenAI on two representative knowledge-work tasks among staff in the organisation. The first task is data related, and the second task is document based. These differ from the types of tasks considered so far in the literature.
Experimental design
A two phase design, with a pre-assessment task to baseline individual subjects and two tasks within the main experiment. The control group receives task instructions, data or documents to work from. The treatment group receives task instructions, data/documents to work from and the assistance of an LLM focused on these types of tasks.
Design details
Cited by
No citation data available for this registration yet.