AI Literacy - Towards Measuring Human Competency in Artificial Intelligence

Abstract

Artificial intelligence (AI) has gained significant traction in information systems (IS) research in recent years. While past studies have identified many effects of AI technology on human-AI collaborations, there is a paucity in IS literature on the competencies of humans that affect this relationship. In this study, we set out to develop a measurement instrument (scale) for general AI literacy, that is humans' socio-technical competencies regarding AI. We conducted a systematic literature review followed by five expert interviews to define and conceptualize the construct of general AI literacy and to generate an initial set of items. Furthermore, we performed two rounds of card sorting with six and five judges and a pre-test study with 50 participants to evaluate the developed scale. The validated measurement instrument contains five dimensions and 13 items. We provide empirical support for the measurement model and conclude with future research directions.