Writing effective autograded exercises using Bloom's taxonomy

Abstract

Computer Science (CS) enrollment continues to grow every year and many CS instructors have turned to auto-graded exercises to ease grading load while still allowing students to practice concepts. As the use of autograders becomes more common, it is important that the exercise sets are being written to maximize student benefit. In this paper, we use Bloom's Taxonomy (BT) to create auto-graded exercise sets that scale up from lower to higher levels of complexity. We conducted a field experiment in an introductory programming course (264 students) and focused on evaluating learning efficiency, code quality, and student perception of their learning experience. We found that it takes students more submission attempts in the auto-grader when they are given BT Apply/Analyze-type questions that contain some starter code. Students complete the auto-graded assignments with fewer number of submissions when there is no-starter code and they have to write their solution from scratch, i.e. BT Create-type of questions. However, when writing code from scratch, the students' code quality can suffer because the students are not required to actually understand the concept being tested and might be able to find a workaround to pass the tests of the auto-grader.