Integrating AI-Based Natural Language Processing in Vocational Education: Usability, Learning Gains, and Student Engagement in Indonesia

Abstract

Introduction: the advancement of Artificial Intelligence (AI) has brought substantial changes to education, particularly through AI-based digital assistants. Objective: this study developed and evaluated an AI-powered digital assistant equipped with Natural Language Processing (NLP) capabilities, specifically designed for Indonesian vocational schools. Method: adopting the 4D development model (Define, Design, Develop, Disseminate), the system was created using machine learning algorithms and NLP to enhance interactivity and personalization. The assistant enables natural language interaction, provides real-time feedback, and adapts learning material difficulty to students’ comprehension levels. The system was tested with 100 vocational school students, with usability assessed using the System Usability Scale (SUS) and learning gains measured through pre-and post-tests. Results: results showed a SUS score of 71,05, indicating good usability, and a significant improvement in post-test scores compared to pre-test scores (p < 0,001), reflecting enhanced conceptual understanding, engagement, and motivation. Conclusions: these findings demonstrate the potential of AI-powered NLP assistants to enrich vocational education and prepare students for technology-driven industrial demands.