Limits and capabilities of LLM
How to cite
Hakimov, Rustamdjan; Bohren, Noah; Lalive, Rafael. 2023. "Limits and capabilities of LLM." AEA RCT Registry. AEARCTR-0011584. https://doi.org/10.1257/rct.11584-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We are conducting an experiment to determine whether state-of-the-art language prediction models can outperform humans in areas of creativity and strategic intelligence. Comparing human performance to these advanced models is of paramount importance. This comparison sheds light on the level of competition that language models can present and the range of their potential applications, especially in areas traditionally seen as beyond their capabilities. Additionally, we also study effect of competition with artificial intelligence on creativity of human subjects, both for those who perform the tasks and for the evaluators, who might alter their perceptions of creativity when informed that some of the responses were generated by AI.
Intervention
In the first experiment, the interventions vary the information about competitors.
In the second experiment, participants rate texts, and interventions vary information about those who submitted the text for ratings.
Experimental design
In the first experiment, Prolific participants undertake creative and strategic tasks. The interventions vary the information about competitors.
In the second experiment, Prolific participants rate creative responses to one of two questions.The interventions vary information about those who submitted the text for ratings.
Design details
Cited by
No citation data available for this registration yet.