"you're on a bicycle with a little motor": Benefits and Challenges of Using AI Code Assistants

Abstract

AI code assistants, such as Tabnine, GitHub CoPilot, and ChatGPT, employ Large Language Models (LLMs) trained on extensive source code and other documents. They receive prompts and generate code suggestions aimed to facilitate programming tasks. Previous research in this field has explored the correctness, complexity, quality, and security of the code suggestions. Software developers' experiences have been studied in the context of controlled experiments. Based on 14 interviews with software developers, this paper describes the developers' daily and continuous experiences with AI code assistants, presenting benefits and challenges grounded in actual development work, along with strategies to address these challenges.