Evaluation Frameworks for Digital Nursing Technologies: Analysis, Assessment, and Guidance

Abstract Background: The evaluation of digital nursing technologies (DNT) plays a major role in gaining knowledge into certain aspects of a technology such as acceptance, effectiveness, or efficiency. Evaluation frameworks can help to classify the success or failure of a DNT or to further develop the technology. In general, there are many different evaluation frameworks in the literature that provide overviews of a wide variety of aspects, which makes this a highly diverse field and raises the question of how to select a suitable framework. The aim of this article is to provide transparency on existing comprehensive evaluation frameworks that can be applied to the field of DNT and to conduct a detailed analysis and assessment of these frameworks to guide field researchers. Methods: A three components search process was conducted to identify relevant frameworks. (1) A systematic literature search in PubMed; a narrative review (2) and (3) expert consultations. Data related to the frameworks evaluation areas, purpose, perspectives, and success definitions were extracted. Quality criteria were developed in an expert workshop and a strength and weakness assessment was carried out. Results: 18 relevant comprehensive evaluation frameworks for DNT have been identified. 9 overarching evaluation areas, 7 categories of purposes, 5 evaluation perspectives and 3 categories of success definitions could be identified. Eleven quality criteria for the strengths and weaknesses of DNT-related evaluation frameworks were developed and the included frameworks were assessed against them.Conclusion: Evaluators can use the concise information and quality criteria of this article as a starting point to select and apply appropriate DNT evaluation frameworks for their research projects or to assess the quality of an evaluation framework for DNT, as well as a basis for exploring the questions raised in this article. Future research could address gaps and weaknesses in existing evaluation frameworks, which could improve the quality of future DNT evaluations.