The role of students’ higher-order thinking skills in the relationship between academic achievements and machine learning using generative AI chatbots

Abstract

Students’ perspectives on using generative artificial intelligence (AI) chatbots and machine learning are crucial in shaping the design, development, and implementation of their learning projects across various disciplines. Cognitive thinking, a key aspect of AI-related machine learning, aims to replicate human intelligence and behavior. However, the relation between cognitive thinking and knowledge acquisition is often overlooked. This cross-sectional study empirically examines the relationship between academic achievement and students’ attitudes toward machine learning, particularly through the use of generative AI chatbots. It specifically focuses on the role of higher-order thinking skills—such as problem-solving, critical thinking, and creativity—as both mediators and moderators in this relationship. A total of four hundred sixteen undergraduate students (n=416) from diverse academic backgrounds voluntarily took part in a project, in which they designed and developed generative AI chatbots in media and information literacy courses. The findings indicate that creativity mediated the relationship between academic achievements and attitudes toward machine learning, but its moderating impact was not significant. Problem-solving and critical thinking did not show significant mediating effects on attitudes toward machine learning, while they showed significant moderating effects in the connection between academic performance and attitudes toward machine learning. This study contributes by elucidating the interrelationships between students’ higher-order thinking skills, academic performance, and attitudes on the use of AI and machine learning technologies. By highlighting the mediating role of creativity and the moderating effects of problem-solving and critical thinking, this study offers a deeper understanding of how these skills shape students’ perceptions of AI.