Measuring student perceptions of AI-assisted academic communication: A questionnaire development study

Abstract

The aim of this study is to develop a valid and reliable scale evaluating students’ views on artificial intelligence (AI) supported academic communication. Knowing how students view these tools is crucial considering AI’s increasing presence in the classroom. We applied a thorough approach comprising content validation, pilot research and factor analysis in addition to literature review. Initially, 40 items were created and reduced to 37 items after expert evaluation. As a result of the analysis of the data obtained from 580 participants, it was determined that the scale showed a two-factor structure as “positive dimension” and “negative dimension”. Factor analyses both exploratory and confirmatory helped to establish the scale’s construct validity. The scale’s internal consistency reliability came out to be really strong. Based on gender and age, Bayesian statistical studies revealed no appreciable variation in students’ opinions of academic communication supported by AI. The developed scale offers academics and teachers a consistent instrument to evaluate students’ perception of AI technologies. This scale will help to shape plans for better integration of AI into learning environments.