Effectiveness of using an intelligent tutoring system to train users on off-the-shelf software

Abstract

This study addresses the potential of using an intelligent tutoring system (ITS) to tutor on off-the-shelf (OTS) software. ITSs have been successfully used to tutor on a variety of learning domains, but there has been little research comparing ITS-based training on an OTS application with traditional software training approaches such as books or interactive software simulations. The work presented here includes procedures and results for training and evaluation using three methods: book-based, interactive simulation, and an ITS. We found that there were some associations between the training method and training experiences. Book-based training exhibited higher scores on both task performance and system usability perception, while better times were recorded for the simulation approach. Concept acquisition score was not found to significantly correlate with training method. We concluded that if an ITS is to be a tutor on OTS applications then further refinements are needed.