Design science research evaluation

Abstract

The consensus view is that the rigorous evaluation of design science (DS) artifacts is essential. There are many types of DS artifacts and many forms of evaluation; what is missing is guidance for how to perform the evaluation, more specifically, what evaluation methods to use with specific DS research outputs. Here we find and review 148 DS research articles published in a selected set of information systems (IS), computer science (CS) and engineering journals. We analyze the articles to develop taxonomies of DS artifact types and artifact evaluation methods; we apply these taxonomies to determine which evaluation methods are associated in the literature with particular artifacts. We show that there are several popular "artifact - evaluation method" combinations in the literature. The results inform DS researchers of usual and customary combinations of research artifacts and evaluation methods, potentially providing them with rationale and justification for an evaluation method selection. © 2012 Springer-Verlag.