Development and Validation of the Systems Thinking Scale

Abstract

Purpose: Systems thinking is the ability to recognize and synthesize patterns, interactions, and interdependencies in a set of activities and is a key component in quality and safety. A measure of systems thinking is needed to advance our understanding of the mechanisms that contribute to improvement efforts. The purpose of this study was to develop and conduct psychometric testing of a systems thinking scale (STS). Methods: The development of the STS included obtaining national quality and safety experts’ conceptual domains of systems thinking and the generation of a provisional set of items. Further psychometric analyses were conducted with interprofessional healthcare faculty (N = 342) and students (N = 224) engaged in quality improvement initiatives and education. Results: Of the 26 items identified in the development phase, factor analyses indicated three factors: (1) system thinking (20 items), (2) personal effort (2 items), and (3) reliance on authority (4 items). The six items from factors 2 and 3 were omitted due to low factor loadings. Test-retest reliability of the 20-item STS was performed on 36 healthcare professionals and a correlation of 0.74 was found. Internal consistency testing on a sample of 342 healthcare professionals using Cronbach’s alpha showed a coefficient of 0.89. Discriminant validity was confirmed with three groups of healthcare professions students (N = 102) who received high, low, or no dose levels of systems thinking education in the context of process improvement. Conclusions: The 20-item STS is a valid and reliable instrument that is easy to administer and takes less than 10 min to complete. Further research using the STS has the potential to advance the science and education of quality improvement in two main ways: (1) increase understanding of a critical mechanism by which quality improvement processes achieve results, and (2) evaluate the effectiveness of our education to improve systems thinking.