The Adoption of Generative Artificial Intelligence
How to cite
Humlum, Anders; Vestergaard, Emilie. 2023. "The Adoption of Generative Artificial Intelligence." AEA RCT Registry. AEARCTR-0012527. https://doi.org/10.1257/rct.12527-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
The arrival of ChatGPT marks the era of Generative Artificial Intelligence (AI), in which intelligent algorithms create new text, images, and other media. ChatGPT is predicted to boost productivity in several occupations (Eloundou et al., 2023), especially among workers with less expertise in the exposed occupations (Noy & Zhang, 2023).
This project asks three questions: Which workers have adopted ChatGPT? What do workers believe about the capabilities of ChatGPT in their job tasks? Do worker beliefs about the capabilities of ChatGPT determine their adoption?
To answer these questions, we conduct a survey with an embedded randomized treatment among Danish workers in 11 occupations exposed to ChatGPT. We link the survey to registry data on the characteristics and outcomes of workers, including their occupation, skill, experience, workplace, and earnings, among others. We build the survey around the task model of technology adoption (Acemoglu & Autor, 2011) and use the Occupational Information Network (O*NET) database to identify the relevant job tasks for each occupation.
We conduct a main survey and a follow-up.
The main survey is organized into five parts. First, we ask workers about their experiences with ChatGPT in their job tasks. We also ask about the importance of these tasks for workers and their expertise in the tasks. Second, we elicit workers' beliefs about the productivity and expertise-complementarity of ChatGPT in their job tasks. Third, we expose our treatment groups to expert assessments of the capabilities of ChatGPT (productivity and expertise-complementarity) in the job tasks. Fourth, we ask workers about their intended use of ChatGPT in their job tasks. We also allow workers to receive additional information on ChatGPT in their occupations. Finally, we ask workers about their beliefs about the productivity of ChatGPT in their own jobs, together with any misalignments in their stated beliefs and intended behaviors.
We conduct a follow-up survey two weeks after the workers' response to the main survey. The follow-up measures whether workers' beliefs persist over time and if workers' intended adoption translates into the actual use of ChatGPT.
We randomize the sampling and treatment by workers and workplaces. The adoption study described in the analysis plan focuses on the worker-level experiment.
Intervention
We will run a survey with a randomized information provision experiment on the capabilities of ChatGPT in workers’ job tasks.
Treatment 1 ("productivity") exposes workers to information about the general productivity of ChatGPT in each job task. Following Eloundou et al. (2023), we inform workers whether access to ChatGPT can reduce the time it takes for an average worker to complete a task by at least half (50%).
Treatment 2 ("complementarity") exposes workers to information about whether the time savings from ChatGPT are smaller, similar, or larger for workers with expertise in that task.
The treatment groups see a comparison of their assessments and the expert assessments, together with brief explanations of the expert assessments. The control group only sees a summary of their prior assessments.
Experimental design
Our experiment exposes workers to expert assessment of the capabilities of ChatGPT in different job tasks. Our expert assessments are based on Eloundou et al. (2023), adapted to the Danish context, and validated by industry specialists.
We have three treatment arms:
Treatment 1 ("productivity"): Workers receive information about the general productivity of ChatGPT in each job task. Following Eloundou et al. (2023), we inform workers whether access to ChatGPT can reduce the time it takes for an average worker to complete a task by at least half (50%).
Treatment 2 ("complementarity"): Workers receive information about whether the time savings from ChatGPT are smaller, similar, or larger for workers with expertise in that task.
Control: The control group only sees a summary of their prior assessments.
The treatment groups see a comparison of their assessments and the expert assessments, together with brief explanations of the expert assessments.
Prior to the treatment, we ask workers about the importance of the job tasks and their expertise in the tasks. We also ask workers about their experiences with ChatGPT in the job tasks. Furthermore, we elicit workers' prior beliefs about the general productivity (treatment 1) and expertise-complementarity (treatment 2) of ChatGPT in the job tasks.
After the treatment, we ask workers about their intended use of ChatGPT in their job tasks. We also allow workers to sign up for additional material on potential use cases of ChatGPT in their occupations.
Finally, we ask workers about their beliefs about the productivity of ChatGPT in their own jobs. We also ask workers about misalignments in their stated beliefs and intended behaviors. The follow-up questions unpack the chain from information provision via updated beliefs to adoption behaviors.
We will conduct a follow-up survey two weeks after the workers' response to the main survey. The follow-up will assess (i) whether workers' beliefs about ChatGPT persist over time, (ii) whether workers' intended adoption of ChatGPT manifests into the actual adoption, and (iii) short-run spillovers across coworkers in belief formation and adoption behaviors.
Additional information on selecting occupations:
We survey occupations that meet the following criteria:
1. The occupation has a well-identified occupational code (ISCO and SOC).
2. The occupation has at least one job task in which ChatGPT can reduce the time it takes for an average worker to complete it by at least half (Eloundou et al., 2023).
3. The occupation has enough workers for statistical analysis.
The resulting list of occupations is:
Accountants
Customer supporters
Financial advisors
HR professionals
IT supporters
Journalists
Legal professionals
Marketing professionals
Software developers
Teachers
Office clerks
Additional information on selecting job tasks:
We use the Occupational Information Network (O*NET) to identify the job tasks for each occupation. For each occupation, we select 6 job tasks that represent the capabilities of ChatGPT ("general productivity" and "expertise-complementarity") in the occupation. For task combinations with similar representation, we prioritize combinations with higher task relevances, as measured by O*NET.
Design details
Cited by
No citation data available for this registration yet.