"it's like a puppet master": User perceptions of personal autonomy when interacting with intelligent technologies

Abstract

Applications which use some form of artificial intelligence (AI) have become embedded in our everyday interactions. Very often, AI-based apps are personalized and modelled on users' needs and preferences. However, such applications of AI tread a delicate balance between enhancing user experience and jeopardizing personal autonomy. Personal autonomy and sense of agency are crucial for human well-being and development. In this paper, we probe this fine balance aiming to capture users' lived experiences and perceptions of interacting with AI-based apps. We present insights from a phenomenological study (N=15) regarding users' perception of personal autonomy when interacting with AI in everyday contexts. We found that these experiences are transitory and largely influenced by contextual factors. Users experience a loss of autonomy when their privacy or identity is threatened or when their expectations are broken. To mitigate such loss of autonomy, mechanisms for providing intelligibility and control of AI are desired.