A study on two hint-level policies in conversational intelligent tutoring systems

Abstract

In this work, we compared two hint-level instructional strategies, minimum scaffolding vs. maximum scaffolding, in the context of conversational intelligent tutoring systems (ITSs). The two strategies are called policies because they have a clear bias, as detailed in the paper. To this end, we conducted a randomized controlled trial experiment with two conditions corresponding to two versions of the same underlying state-of-the-art conversational ITS, i.e. DeepTutor. Each version implemented one of the two hint-level strategies. Experimental data analysis revealed that pre-post learning gains were significant in both conditions. We also learned that, in general, students need more than just a minimally informative hint in order to infer the next steps in the solution to a challenging problem; this is the case in the context of a problem selection strategy that picks challenging problems for students to work on.