Is that realistic? The development of a realism assessment questionnaire and its application in appraising three simulators for a gynaecology procedure

Abstract

Introduction: There is no standard approach to determining the realism of a simulator, valuable information when planning simulation training. The aim of this research was to design a generic simulator realism questionnaire and investigate the contributions of different elements of simulator design to a user’s impression of simulator realism and performance. Methods: A questionnaire was designed with procedure-specific and non-procedure-specific (global) questions, grouped in subscales related to simulator structure and function. Three intrauterine contraceptive device (IUCD) simulators were selected for comparison. Participants were doctors of varying experience, who performed an IUCD insertion on each of the three models and used the questionnaire to rate the realism and importance of each aspect of the simulators. The questionnaire was evaluated by correlation between procedure-specific and global items and the correlation of these items to overall realism scores. Realism scores for each simulator were compared by Kruskal-Wallis and subsequent between-simulator comparison by Dunn’s test. Results: Global question scores were highly related to procedure-specific scores. Comparison revealed global item subscale scores were significantly different across models on each of the nine subscales (P < 0.001). Function items were rated of higher importance than structure items (mean function item importance 5.36 versus mean structure item importance 5.02; P = 0.009). Conclusions: The designed questionnaire was able to discriminate between the models for perceived simulator realism. Findings from this study may assist simulator design and inform future development of a generic questionnaire for assessing user perceptions of simulator realism.