Towards a comprehensive online peer assessment system: Design outline

Abstract

The business of business education is rapidly evolving because of changing economic and social conditions. At many institutions, class sizes are growing, more curricula is being offered online and traditionally successful pedagogical standards are being threatened unless they are adapted to the emerging economic realities of the 21st century. In response to the economic threats and consumers' preferences, numerous IT artifacts are being created to facilitate online teaching in the hope that both quality and cost concerns will be ameliorated (albeit not always using the precepts of design science). The purpose of this 'work-in-progress' paper is to apply design science principles to outline an algorithm for a computer-aided peer assessment system, named Double-loop Mutual Assessment (DLMA). The project's goal is to emulate the case method online, improve students' learning experience and increase grading efficacy. The DLMA yields two IT artifacts: a method and an instantiation. The DLMA method artifact involves two loops of assessment: 1) a summative and formative mutual peer assessment algorithm for essays; and 2) a summative peer assessment of the feedback's quality. An instantiation of DLMA system - a prototype and a beta-version has been implemented and described. Future directions of researching behavioral and operational aspects of the system are outlined. Potential applications of the artifact's capabilities beyond the business necessity are discussed. © 2012 Springer-Verlag.