Behavioral measures improve AI hiring: A field experiment
Pages 831 - 832
Abstract
The adoption of Artificial Intelligence (AI) for hiring processes is often impeded by a scarcity of comprehensive employee data. We hypothesize that the inclusion of behavioral measures elicited from applicants can enhance the predictive accuracy of AI in hiring. We study this hypothesis in the context of microfinance credit officers. We focus on cognitive and behavioral measures commonly used in economics and psychology.
We first investigate the efficiency of behavioral measures to explain the productivity of the existing employees. All employees of the firm take a 90-minute survey composed of incentivized and non-incentivized measures. We then use a sample of employees with a minimum tenure of 12 months to train an algorithm to predict which employees will receive a bonus. A random forest algorithm, relying solely on firm data such as age, education level, marital status, and other factors, accurately categorizes 65% of employees in an out-of-sample test. The inclusion of non-incentivized survey measures significantly improves the performance of the random forest algorithm by five percentage points.
There are two concerns about using survey measures for hiring decisions. First, applicants might alter their responses to fit what they believe is the company's ideal employee profile. Second, our training sample includes only employees who worked for the company for over a year, which might bias the algorithm.
As a first step to address the issue of on-the-job selection, we provide an out-of-sample prediction for employees with less than one year of tenure at the time of the survey. We examine their propensity to receive a bonus both at the 12-month mark of their employment and one year after the survey. Our findings indicate that those predicted by the algorithm to be productive are significantly more likely to receive a bonus at both time horizons, conditional and unconditional on being employed.
To further address selection into the job and potential strategic responses, we conduct a field experiment in which all job applicants were evaluated by the algorithm. We divide applicants into two groups: HR and AI treatments. In the HR treatment, local and regional managers made hiring decisions as usual, but we also recorded the algorithm's recommendation. In the AI treatment, a decision by local and regional managers was overridden if it conflicted with the algorithm's recommendation.
We compare the performance of employees hired under these two treatments at the 12-month mark of their employment and in February 2024. Overall, the performance of employees hired in the AI treatment is better than in the HR treatment. However, the differences are only marginally significant. We also examine the predictive power of the algorithm. Those who were recommended for hiring by the algorithm are significantly more likely to receive a bonus at the 12-month mark and in February 2024 compared to those who were not recommended. Based on these findings, we conclude that the algorithm is robust to both selection effects and strategic manipulation of responses in our context.
Index Terms
- Behavioral measures improve AI hiring: A field experiment
Index terms have been assigned to the content through auto-classification.
Recommendations
Algorithmic Hiring in Practice: Recruiter and HR Professional's Perspectives on AI Use in Hiring
AIES '21: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and SocietyThe use of AI-enabled hiring software raises questions about the practice of Human Resource (HR) professionals' use of the software and its consequences. We interviewed 15 recruiters and HR professionals about their experiences around two decision-...
IT Hiring Growth Modest, But Steady
IT hiring and salaries are expected to increase in 2006, but to thrive in the IT marketplace, employees will need to demonstrate an expertise in more than just a programming language. Soft skills, such as Cobol and SAP can be key differentiators. ...
A framework of the use of certifications by hiring personnel in it hiring decisions
IT certification represents a common and growing practice with significant implications for managing in-house IT workers and outsourcing contracts. Prior research is of limited value in understanding its use in common personnel decisions, such as hiring,...
Comments
Information & Contributors
Information
Published In
July 2024
1340 pages
ISBN:9798400707049
DOI:10.1145/3670865
- Chair:
- Dirk Bergemann,
- Program Chairs:
- Robert Kleinberg,
- Daniela Saban
Copyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 December 2024
Check for updates
Qualifiers
- Extended-abstract
Funding Sources
- Swiss National Science Foundation
Conference
EC '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 664 of 2,389 submissions, 28%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 48Total Downloads
- Downloads (Last 12 months)48
- Downloads (Last 6 weeks)19
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in