ASSIST: Automated Feedback Generation for Syntax and Logical Errors in Programming Exercises

Abstract

Introductory programming courses often rely on numerous exercises to help students practice and reinforce their skills. Commonly used automated tests fall short by merely identifying the issues without offering guidance on how to resolve them and manual reviews are too resource-intensive to use in large classes. To address these challenges, we present ASSIST—a tool designed to provide automated, detailed feedback on how to resolve issues in programming exercise submissions with both syntactic and logical errors. ASSIST combines fault-tolerant parsing with fixes based on the context of error nodes to resolve syntactic errors and give feedback. ASSIST feeds this valid program to the Sketch program synthesis tool to determine the needed changes from a set of potential changes induced by rewrite rules, and generates feedback on logic errors based on the needed changes. This dual approach allows ASSIST to offer actionable feedback on both syntax and logic issues in student submissions. We evaluated ASSIST on submissions from an online platform for secondary education. Our findings reveal that, for submissions with syntax errors, ASSIST delivers feedback on all syntax errors in 71% of cases and extends its feedback to cover logical errors in 34% of these submissions. When evaluating all incorrect submissions, ASSIST is able to give feedback on logical errors in 64% of cases. These results indicate that ASSIST can significantly enhance the feedback process in large-scale programming courses, offering a feasible and efficient alternative to current methods.