AI-Enhanced Adaptive Testing: Integrating Response Time with IRT Models

Abstract

AI use in the education system has been changing typical ways of testing students and offering alternative methods of personalized and effective assessments. This study aims to develop a testing system that is innovative and more precise and evaluates the performance of students not only based on right or wrong answers but it incorporates response time and test completion time. Unlike prior adaptive systems that rely solely on response accuracy, this study uniquely integrates response time into the scoring process through cubic and linear regression techniques, combined with AI-based classification models such as Random Forest and kNearest Neighbors (k-NN). This fusion provides a more granular, equitable, and time-aware framework for evaluating student performance. A web-based system, "TestYourself," was developed in Python to implement this adaptive testing approach. The system collects data on answer times, time for completing the test, and correct/incorrect responses for every question. The data is then processed using algorithms of machine learning, where the score of each student is calculated by combining cubic and linear regression, accounting for the difficulty of the question and the time taken for the response. This innovative assessment method has been tested on university students in their first year. The results demonstrate that the AUROC value for classification using the Random Forest algorithm, incorporating both ability scores and time-based extra scores, is 11.13% more sensitive than classification using only ability scores and extra time-based scores, and 33.46% more sensitive than using only ability scores. The new time-based exam assessment and grading system allows for more accurate determination of student abilities in timed exams.