Recruitment Screening with Human and Artificial Intelligence
How to cite
Yanagizawa-Drott, David; Krenk, Ursa; Awuah, Kobbina. 2023. "Recruitment Screening with Human and Artificial Intelligence." AEA RCT Registry. AEARCTR-0011651. https://doi.org/10.1257/rct.11651-1.1
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
This study aims to investigate the potential role of artificial intelligence (AI) in assisting in the screening for labor in the recruitment process, with the use of generative large language models.
We will conduct a randomized controlled trial (RCT) in assisting a non-profit organization in recruiting talent – teachers – within the education sector in a sub-Saharan country. The experiment will involve human evaluators who assess incoming applications for the NGO's fellowship program, with or without the assistance of AI. The human evaluators are tasked to screen applications and decide who continues to the next in-depth interview phase, based on a set of criteria deemed desirable to the organization. We give the same task to the generative AI using prompts. We will explore the disagreement between human and AI in evaluating applications, the human behavioral responses under the AI-assistance in terms of effort and application grading, and the quality of candidates among those that are passed onto the interview phase.
Applications will be randomized into four treatment groups: Human-only (control); Human with AI score assistance; Human with AI score and rationale assistance, and; AI-only. The treatment group will determine whether the evaluator will receive AI assistance in scoring an application and what score (AI-generated or evaluator-determined) will be used to select applicants. In addition to behavioral responses by human evaluators when assisted, we will compare the efficiency of AI-only and AI-assisted recruitment to human-only screening, focusing on the quality of selected applicants, screening costs, and speed.
Intervention
The intervention consists in providing evaluators - people who assess incoming applications - with AI assistance. AI assistance can take two forms: providing them with the scores that the AI would have predicted for a given application, or providing them with the scores and an AI-generated explanation for the choice. The AI-only condition assesses applications without human grading and decisions.
Experimental design
All grading will take place on the Qualtrics platform. We randomize applications into four treatment groups: Human-only decisions (control); Human decisions with AI score assistance; Human decisions with AI score and rationale assistance, and; AI-only decisions. Thus, the treatment group will determine two things: (1) whether the evaluator will receive AI assistance in scoring an application and (2) what score (AI-generated or evaluator-determined) will be used to select applicants. The human evaluators will be informed whether and what type of AI assistance will be given, at the beginning of an application. If assistance is given, it will be provided after the initial score is entered by the human evaluator. Once entered, the information from the AI assistance will be revealed and the human evaluator will be asked for a final score.
Design details
Cited by
No citation data available for this registration yet.