Exploring the Potential of Generative AI to Support Non-experts in Learning Analytics Practice

Abstract

Generative AI (GenAI) has the potential to revolutionize the analysis of educational data, significantly impacting learning analytics (LA). This study explores the capability of non-experts, including administrators, instructors, and students, to effectively use GenAI for descriptive LA tasks without requiring specialized knowledge in data processing, visualization, or programming. Through a laboratory experiment, participants with varying levels of expertise in data analysis engaged in three tasks with different levels of difficulty using ChatGPT. The findings reveal that while there is a small effect of previous expertise on performance, novices and experts achieved remarkably similar scores. Additionally, the study identifies that action sequence variables, such as the sequence’s complexity and the presence of specific actions such as evaluating and checking results, significantly predict performance. These results suggest that while current GenAI technologies are not yet ready to fully support non-experts, they hold the promise of supporting stakeholders, regardless of their technical background, to perform descriptive data analysis in the context of LA practice. This research seeks to start a discussion within the LA community about leveraging AI to scale and expand LA practices, potentially transforming how educational stakeholders engage with and benefit from LA.