Algorithm-facilitated discrimination: a socio-legal study of the use by employers of artificial intelligence hiring systems

Abstract

Artificial intelligence (AI) hiring systems (AHSs) are used by employers every day to screen and shortlist job candidates. Despite this, substantial gaps exist in our understanding of the real – as opposed to theoretical – risks of discrimination when these systems are deployed. This article reports on findings from qualitative empirical research investigating the use of AHSs by Australian employers. It demonstrates that the way in which these systems are operated in practice creates serious risks of algorithm-facilitated discrimination. This may arise from the data, the use of proxies, the system's implementation, new structural barriers, a failure to provide reasonable adjustments, or the facilitation of intentional discrimination. These findings are significant, make an original contribution to an emerging field of research, and are relevant in any jurisdiction where an AHS is in use. There is a lot at stake when such discrimination occurs; as one research participant acknowledged, a ‘job application is literally a person's attempt to change their life with a new job’.