Finding the best employees: A field experiment in hiring

How to cite

Dargnies, Marie-Pierre; Kübler, Dorothea; Hakimov, Rustamdjan. 2021. "Finding the best employees: A field experiment in hiring." AEA RCT Registry. AEARCTR-0008219. https://doi.org/10.1257/rct.8219-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Marie-Pierre Dargnies* , Dorothea Kübler , Rustamdjan Hakimov

* Primary investigator

Registry ID
AEARCTR-0008219
DOI
https://doi.org/10.1257/rct.8219-1.0
First registered
Last updated
Status
In development
Trial dates
2021-09-23 – 2022-09-30
Countries
Kyrgyzstan ()
Registry record
http://www.socialscienceregistry.org/trials/8219

Abstract

Can AI outperform current HR practices in selecting employees for the position of credit salesman? What is the role of economic preferences versus psychological types in explaining the heterogeneity of the employees’ performance?

We will run a field experiment in a microfinance company in Kyrgyzstan. In the first stage, current employees of the firm will answer survey questions to measure a number of economic preferences and psychological traits. We will anonymously match the responses to the survey questions to the personnel data of the firm including a measures of the productivity of the employees. We will train an AI algorithm to predict which employees perform best. The next stage will consist in applying the algorithm to study its usefulness for hiring decisions. Some employees will be hired following the existing procedure in place in the firm, others accodring to the recommendation of the AI algorithm and we will compare the results of the two groups of employees in term of productivity, size and risk of the portfolio.

Intervention

We will have the current employees of a microfinance company in Kyrgyzstan answer a questionnaire, measuring their risk and time preferences, trust and trustworthiness, altruism, Big 5 personality traits, performance in the Cognitive Reflection Test (CRT) and Wonderlic Test, performance in Reading the Mind in the Eyes, and confidence. Some of the measures (risk and time preferences, for instance) are incentivized, while others are not. We will match the responses to the personnel data of the firm using firm's data (anonymized matching). This data includes age, gender, education, family status, tenure, portfolio of loans, quality of the portfolio, and bonus received in the last 12 months whih is a measure of productivity. We will train an AI algorithm to predict which employees perform best according to the rankings based on productivity, portfolio size, and the risk of the portfolio.

The next stage will consist in applying the algorithm to study its usefulness for hiring decisions. All candidates who apply to the firm will participate in the survey, excluding the incentivized measures . As is the case now, without our experimental treatment, all candidates will undergo an interview with the central office and one of the regional managers. At this point, we will be able to see the extent to which the pool of employees recommended by the managers and by the algorithm overlap. Then half of the candidates will be hired based on the existing procedure in the company, that is according to the managers' recommendation. The other half of the candidates will be hired based on the AI prediction of the similarity of the candidates to the best performing employees. We will evaluate the results based on the firm’s record of the performance of the new employees, including their sales, the risk of their portfolio, and whether the employee was fired for underperformance or left the company. The field experiment will run for six months, with the possibility of prolonging it, in case that the number of new hires is too small to draw any conclusions. We aim for at least 100 new hires in each treatment.

Experimental design

All candidates for a salesman position in the firm will, as is the case now (before our intervention), undergo an interview with the central office and one of the regional managers. All candidates will also participate in our survey.
Then, half of the candidates will be hired based on the existing procedure in the company, that is according to the managers' recommendation. The other half of the candidates will be hired based on the AI prediction of the similarity of the candidates to the best performing employees.

Design details

Randomization method
Randomization will be done by a computer, with every second candidate being in treatment.
Randomization unit
Individual.
Clusters
No clusters are planned.
Observations
At least 200 new employees. In order to train the AI algorithm, we will use the answer to the survey of the current employees of the firm (about 1000 employees).
Arms
At least 100 new employees in each treatment (hired via the existing procedure/ hired using the AI algorithm)
Primary outcomes
The primary outcome of the experiment is the performance of the new employees. We are interested in whether the performances of the new employees are different whether the new employees were hired using the existing hiring procedure of the firm or the AI algorithm recommendation. For the first stage the main outcomes are performances of the algorithm given personally data only, personnel data and non-incentivized survey measures, and personnel data , non-incentivized and incentivized survey measures. Portfolio size Portfolio at risk 30 days or more Portfolio free of risk Sales We are interested in outcomes in 6 and 12 months after starting the job. We train algorithm on employees with tenure of 12 months or more. This is because the categorization by management is more informative, as well as their objective performance. We will test the algorithm predictive power first of the employees with lower than 12 month of tenure in the moment. We will predict the category for them, and evaluate based on the performance after 6 and 12 month from start of the job.

Cited by

No citation data available for this registration yet.

Keywords