Gamebrics: Design, implementation and practical evaluation of analytical rubrics in serious game play

Abstract

Background: Complex skills, like analytical thinking, are essential in preparing students for future professions. Serious games hold potential to stimulate the online acquisition of such professional skills in an active and experiential way. Objective: Rubrics are proven assessment and evaluation instruments, but were never directly integrated into actual gameplay. We present our approach towards the integration of analytical rubrics into gameplay, the implementation into existing (scenario-based) serious games, and main findings from a study on learning effects and appreciation of the tooling (dashboard with progress information and reflective feedback). Method: Our experimental research setup randomly allocated 84 participating students into either a control group or an experimental group. The intervention for the control group consisted of an existing serious game with natural feedback as part of the scenario, supporting the gameplay itself. The intervention for the experimental group also provided an additional dashboard with reflective feedback, monitoring the acquisition of analytical skills through gameplay. Learning growth through gameplay was measured by pre-/post-test scores on a knowledge test, in-game performance by means of computer logging, and appreciation of the dashboard and feedback by means of a questionnaire. Results and Conclusions: Participants receiving reflective feedback show higher learning increases (on a pre-/post-knowledge test), and more efficient gameplay behaviour (on performance scores). Participants receiving reflective feedback feel more confident about competence development. These findings appear promising for this approach on integrating formative assessment in serious gameplay to support the acquisition of complex skills.