The Use of ChatGPT Applications in Learning: Impact on Understanding and Student Engagement in TVET Institutions

Abstract

In the field of Technical and Vocational Education and Training (TVET), the use of artificial intelligence (AI) technology has the potential to enrich the learning experience. However, the challenge lies in determining the level of skills required by students to optimize the learning experience and enhance understanding of technical concepts and student engagement. The main objective of this study is to examine the relationship between students' understanding and engagement based on their skill levels in using ChatGPT application in the TVET education sector. A pilot study was conducted with 30 respondents for all 24 items to measure the reliability of the research instrument. The findings indicated a Cronbach's alpha value of 0.982, suggesting that all constructed items are suitable for use as a measurement tool in assessing the needs of the developed survey instrument. Independent sample t-test analyses were conducted to determine if there were significant differences in understanding and student engagement based on students' skills in using the ChatGPT application. A total of 173 respondents participated in this study, comprising 125 students from the Information and Communication Technology Department (ICTD) and 48 students from the Design and Visual Communication Department (DVCD) at Polytechnic Besut Terengganu (PBT). Data obtained from the survey were analyzed using Statistical Packages for Social Sciences version 26.0. Descriptive analysis of minimum comparisons showed a positive impact, indicating that students in TVET institutions agree that the use of the ChatGPT application in learning influences their level of understanding and engagement based on high minimum values. However, independent sample t-test analysis showed differing results. While there was no significant difference in understanding based on the skills of using the application, there was a significant difference in student engagement. This suggests that although students consider the ChatGPT application to have an overall positive impact, the benefits are consistent with students' understanding without making a difference to a specific group. Skills in using this application have a more significant impact on student engagement than on their understanding. Despite the overall positive impact of the application, a focus on student engagement may be a key factor in enhancing the effectiveness of this application in the TVET learning context. To improve the effectiveness of the ChatGPT application, further research on factors contributing to differences in students' engagement levels needs to be conducted. Further studies could also involve comparative research between TVET institutions to identify differences in perceptions and the use of the ChatGPT application. Long-term studies to measure the impact of the ChatGPT application on academic achievement could provide a deeper understanding of its long-term effectiveness in facilitating the learning process in TVET institutions. Opportunities and challenges related to AI in education should also be investigated to support and expand these findings in various technical disciplines and the TVET education context.