Evaluation frameworks for digital nursing technologies: analysis, assessment, and guidance. An overview of the literature

Abstract

Background: The evaluation of digital nursing technologies (DNT) plays a major role in gaining knowledge about certain aspects of a technology such as acceptance, effectiveness, or efficiency. Evaluation frameworks can help to classify the success or failure of a DNT or to further develop the technology. In general, there are many different evaluation frameworks in the literature that provide overviews of a wide variety of aspects, which makes this a highly diverse field and raises the question how to select a suitable framework. The aim of this article is to provide orientation in the field of comprehensive evaluation frameworks that can be applied to the field of DNT and to conduct a detailed analysis and assessment of these frameworks to guide field researchers. Methods: This overview was conducted using a three-component search process to identify relevant frameworks. These components were (1) a systematized literature search in PubMed; (2) a narrative review and (3) expert consultations. Data relating to the frameworks’ evaluation areas, purpose, perspectives, and success definitions were extracted. Quality criteria were developed in an expert workshop and a strength and weakness assessment was carried out. Results: Eighteen relevant comprehensive evaluation frameworks for DNT were identified. Nine overarching evaluation areas, seven categories of purposes, five evaluation perspectives and three categories of success definitions could be identified. Eleven quality criteria for the strengths and weaknesses of DNT-related evaluation frameworks were developed and the included frameworks were assessed against them. Conclusion: Evaluators can use the concise information and quality criteria of this article as a starting point to select and apply appropriate DNT evaluation frameworks for their research projects or to assess the quality of an evaluation framework for DNT, as well as a basis for exploring the questions raised in this article. Future research could address gaps and weaknesses in existing evaluation frameworks, which could improve the quality of future DNT evaluations.