Using cloze procedure questions in worked examples in a programming tutor

Abstract

In order to increase the engagement of learners, we incorporated cloze procedure questions into the worked-example-style feedback provided by problem-solving tutors currently used by introductory programming students unsupervised. We conducted a multi-institution controlled study to evaluate the effectiveness of this intervention from fall 2012 through spring 2014. The results of the study were mixed. We found that when students had to answer cloze procedure questions embedded in the feedback, they did spend significantly more time per problem and they learned concepts with significantly fewer practice problems. However, they did not learn significantly more concepts and their change in score from pretest to post-test was not any different on the learned concepts from that of control group. Finally, the increased time on task due to the intervention may benefit different demographic subgroups differently.