Workshops as a research method: Guidelines for designing and evaluating artifacts through workshops

Abstract

Workshops are often used in the information systems (IS) and design fields to evaluate artifacts or to co-create business innovations. However, the evaluation of workshops is often conducted in a rather unsystematic and heterogenous way. This paper introduces a set of guidelines for designing or evaluating artifacts through workshops. These guidelines include five evaluation principles and a framework that outlines appropriate evaluation methods for different research goals. The relevant constructs and principles were identified based on related literature. The derived evaluation matrix was then revised based on ratings of five experts who independently assigned appropriate research methods for different evaluation foci. The framework's applicability was evaluated by comparing it with ten papers from the IS and design fields. The proposed guidelines can support researchers with conducting workshop evaluations in a comparable and replicable way, which will help to improve research rigor in the future.