Artifact Evaluation in Information Systems Design Science Research ? A Holistic View

Design science in Information Systems (IS) research pertains to the creation of artifacts to solve real-life problems. Research on IS artifact evaluation remains at an early stage. In the design-science research literature, evaluation criteria are presented in a fragmented or incomplete manner. This paper addresses the following research questions: which criteria are proposed in the literature to evaluate IS artifacts? Which ones are actually used in published research? How can we structurethese criteria? Finally, which evaluation methods emerge as generic means to assess IS artifacts? Theartifact resulting from our research comprises three main components: a hierarchy of evaluationcriteria for IS artifacts organized according to the dimensions of a system (goal, environment,structure, activity, and evolution), a model providing a high-level abstraction of evaluation methods,and finally, a set of generic evaluation methods which are instantiations of this model. These methods result from an inductive study of twenty-six recently published papers