Evaluating the shoulder toolkit impact on usability, workload, and confidence in novice and expert ultrasound users

Abstract

Introduction: Point-of-care ultrasound (POCUS) is widely used for musculoskeletal assessment, yet novice users often lack confidence and consistency. The Shoulder Toolkit (STK) was developed to provide a structured framework for shoulder ultrasound. This study compared the STK with traditional methods among novice and expert users. Methods: Thirty-three participants (17 novices, 16 experts) performed standardized shoulder examinations using both traditional methods and the STK. Outcomes included participant confidence, satisfaction, usability (System Usability Scale, SUS), user experience (UXS), net promoter score (NPS), workload (NASA-TLX), and image quality (5-point scale). Assessments were conducted pre- and post-examination. Results: Novices demonstrated significant confidence gains with the STK compared to both pre-examination and traditional methods (p < 0.001). Novice satisfaction was higher with the STK (p = 0.002), whereas experts preferred traditional methods (p = 0.023). Usability increased for all users with the STK (p = 0.003), with novices showing marked improvements (p < 0.001). Experts rated traditional usability higher than novices (p = 0.010), though no group differences were observed for the STK. Novices expressed stronger preference for the STK across learnability, efficiency, utility, and satisfaction (all p < 0.005), with higher likelihood to recommend (p = 0.003). NASA-TLX scores indicated lower mental demand with the STK (p < 0.001), alongside performance and frustration differences between novices and experts under traditional and STK conditions. Experts achieved higher image quality scores than novices (p < 0.001). Conclusion: The STK enhanced novice confidence, satisfaction, usability, and reduced workload compared to traditional methods, while experts favored traditional examination. Implications for practice: The STK offers a structured, user-friendly approach that may accelerate novice skill acquisition and promote consistency in musculoskeletal ultrasound, supporting wider adoption of POCUS in clinical practice.