Leveraging a domain ontology to increase the quality of feedback in an intelligent tutoring system

Abstract

Tutoring systems typically contain or generate a set of approved solutions to problems presented to students. Student solutions that don't match the approved ones, but are otherwise partially correct, receive little acknowledgment as feedback, stifling broader reasoning. Additionally, feedback mechanisms rely on having the student model, which requires extensive effort to build. This paper provides an alternative to the traditional ITS architecture by using a hint generation strategy that bypasses the student model and instead leverages off of the domain ontology. Concept hierarchy and co-occurrence between concepts in the domain ontology are drawn upon to ascertain partial correctness of a solution and guide student reasoning towards the correct solution. We describe the strategy incorporated in a tutoring system for medical PBL, wherein the widely available UMLS is deployed as the domain ontology. Evaluation of expert agreement with system generated hints on a 5-point likert scale resulted in an average score of 4.44 (r = 0.9018, p < 0.05). Hints containing partial correctness feedback scored significantly higher than those without it (Wilcoxon Rank Sum, p < 0.001). © Springer-Verlag Berlin Heidelberg 2010.