Unrestricted Use of LLMs in a Software Project Course: Student Perceptions on Learning and Impact on Course Performance

Abstract

Large language models (LLMs) provide round-the-clock personalized programming assistance, unlike course instructors or traditional online information sources such as Stack Overflow. While LLMs can aid in code generation, concerns about over-reliance and the impact on learning persist. This study discusses students’ experiences with LLMs in a software project course where students were allowed to use LLMs freely except for unit test generation. We conducted surveys during course instances in autumn 2023 and spring 2024. The surveys assessed the extent of LLM usage, methods of application, and perceived impact on learning. Results indicate diverse usage patterns, with many students finding LLMs beneficial for efficiency and problem-solving, though over-reliance and poor-quality outputs were noted concerns. The usage patterns can be linked to course performance and time spent on the project.