Best practices for using algorithmic calculated questions via a course learning management system

Abstract

Textbook publishers have created online algorithmic problem banks for books that have high enrollment. These banks allow an instructor to assign problems from the book with students submitting their answers via an online interface. The algorithmic problems ensure that each student gets a different combination of problem parameter values. Problems are graded automatically with partial credit and immediate feedback available. Instructors benefit by not needing to grade the problems. Students benefit by potentially having multiple attempts to solve each problem with feedback in between attempts. However, these online resources are only available for large enrollment courses where it is financially feasible for publishers to create them, and there is normally an extra cost to the students for access. For instructors teaching courses without such publisher resources or for those wanting additional assignments outside the publisher systems, many commonly used Learning Management Systems (LMS) have similar functionality. A number of the LMS packages in current use such as Blackboard", Moodle", Brightspace", and Canvas" have the capability for creating calculated quiz questions with algorithmic features. This allows an instructor to design an online question with a range of values for one or more parameters in a problem such that each attempt will have a different correct answer. This paper presents best practices for designing and using algorithmic calculated questions for quizzes and/or homework. The paper discusses ways to build the questions in advance, test possible answer combinations, and design likely wrong answers for partial credit and feedback. The pros and cons of using these calculated questions are reviewed. Examples and actual experiences from using the questions demonstrate that this is a beneficial way for instructors to enrich the learning experience while streamlining grading. This is an efficient way for a new engineering educator to gradually build a set of automated problems that can be modified to create new problems with minimal additional effort.