On the use of experiments in design science research: A proposition of an evaluation framework

Abstract

Although experiments are considered a valid scientific method for evaluating the outcome of design science research (DSR), only minimal procedural guidelines or standards exist that help researchers in the setup and conduct. To take advantage of and encourage researchers to include design experiments as an inherent part of their build and test cycle, this study proposes a set of guidelines. In order to get a broad overview of how researchers currently apply the experimental method in DSR and to detect potential drawbacks, an extensive review of the extant literature was conducted. On this basis, we propose an evaluation framework that complements the general design science research guidelines of Hevner and colleagues. The purpose of this framework is to assist researchers, reviewers, editors, and readers in understanding possible pitfalls as well as to ask the right questions which need to be answered in the conduct of design experiments. © 2014 by the Association for Information Systems.