Evaluating the effectiveness of a tutorial dialogue system for self-explanation

Abstract

Previous research has shown that self-explanation can be supported effectively in an intelligent tutoring system by simple means such as menus. We now focus on the hypothesis that natural language dialogue is an even more effective way to support self-explanation. We have developed the Geometry Explanation Tutor, which helps students to state explanations of their problemsolving steps in their own words. In a classroom study involving 71 advanced students, we found that students who explained problem-solving steps in a dialogue with the tutor did not learn better overall than students who explained by means of a menu, but did learn better to state explanations. Second, examining a subset of 700 student explanations, students who received higherquality feedback from the system made greater progress in their dialogues and learned more, providing some measure of confidence that progress is a useful intermediate variable to guide further system development. Finally, students who tended to reference specific problem elements in their explanations, rather than state a general problem-solving principle, had lower learning gains than other students. Such explanations may be indicative of an earlier developmental level. © Springer-Verlag 2004 References.