Exploring Students' Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey

Abstract

ChatGPT has sparked both excitement and skepticism in education. To analyze its impact on teaching and learning it is crucial to understand how students perceive ChatGPT and assess its potential and challenges. Toward this, we conducted a two-stage study with senior students in a computer engineering program (n=56). In the first stage, we asked the students to evaluate ChatGPT using their own words after they used it to complete one learning activity. The returned responses (3136 words) were analyzed by coding and theme building (36 codes and 15 themes). In the second stage, we used the derived codes and themes to create a 27-item questionnaire. The students responded to this questionnaire three weeks later after completing other activities with the help of ChatGPT. The results show that the students admire the capabilities of ChatGPT and find it interesting, motivating, and helpful for study and work. They find it easy to use and appreciate its human-like interface that provides well-structured responses and good explanations. However, many students feel that ChatGPT's answers are not always accurate and most of them believe that it requires good background knowledge to work with since it does not replace human intelligence. So, most students think that ChatGPT needs to be improved but are optimistic that this will happen soon. When it comes to the negative impact of ChatGPT on learning, academic integrity, jobs, and life, the students are divided. We conclude that ChatGPT can and should be used for learning. However, students should be aware of its limitations. Educators should try using ChatGPT and guide students on effective prompting techniques and how to assess generated responses. The developers should improve their models to enhance the accuracy of given answers. The study provides insights into the capabilities and limitations of ChatGPT in education and informs future research and development.