Testing the usability of a software for geospatial and transport modeling in acute stroke service planning

Abstract

Introduction: Geographic visualizations have been used to understand disease since the nineteenth century. We developed a software that creates simple visualizations which can be used as a decision support tool for pre-hospital acute stroke transportation planning. In this study, we test the usability of this software to improve user experience and assess the interpretability of the visualizations it produces as it relates to planning and optimizing stroke systems of care. Materials and Methods: Healthcare practitioners and administrators working within the acute stroke system in Alberta, Canada were invited to participate. Participants were randomized to either the geographic visualization or 2-D temporospatial diagrams. Using a standardized script participants were asked to complete tasks and interpret the visualizations produced by the software. The computer screen and audio were recorded. Recordings were transcribed verbatim and analyzed using inductive thematic analyses. The number of errors made and time to task completion were also analyzed. Results: Eighteen participants (8 physicians, 5 healthcare administrators, 3 paramedics, and 2 nurses) were enrolled. Mean age was 41.22 years (SD: 10.55) and 8 participants were female. It took users a mean of 1.59 min (SD: 0.71) to complete all 10 tasks for the geographic visualizations and a mean of 1.08 min (SD: 0.33) to complete all 15 tasks for the 2-D temporospatial diagrams. Map users made a median of 2 errors (IQR: 4), 2-D temporospatial diagram users also made a median of 2 errors (IQR: 1.5). All but one map user correctly interpreted all maps, only three of the eight 2-D temporospatial diagram correctly interpreted all diagrams. In the qualitative analysis three common themes were identified: Comments on the user interface, comments on the visualization tool(s), and suggestions for improvement. Most study participants mentioned that the software would be useful in their work. Conclusions: Healthcare professional from several different aspects of stroke care see geographic visualizations in transport decision making to be a useful tool. The software demonstrated high usability. However, several suggestions were made to improve user experience as well as additional features which could be developed and become the subject of future studies.