PD27-03 USE OF MACHINE LEARNING ALGORITHMS TO CLASSIFY SURGICAL MANEUVERS

Abstract

INTRODUCTION AND OBJECTIVES: Technology allowing automatic interpretation of complex surgical tasks have wide ranging applications in teaching, robotics and cloud-based computing. Computer vision recognition of surgical maneuvers (e.g. suturing, tying, etc.) can expedite video review and objective analysis. Here, we explore how common machine learning techniques can predict surgical maneuvers from a continuous video record of surgical benchtop simulations. METHOD(S): Hand movements of 37 clinicians performing simple and running subcuticular suturing benchtop simulations without sensors were recorded via digital video. Three machine learning techniques (decision trees, random forests, and hidden markov models) were selected to classify surgical maneuvers every two seconds (60 frames) of video. Twenty percent of all video segments were randomly selected to serve as a test set for random forest prediction across all tasks, while thirty percent of participants from each experience group (medical students, junior residents, senior residents, attendings) comprised a testing set of twelve participants (ie, participant-controlled). This allowed for both random selection and within-user population accuracy prediction estimates. RESULT(S): Applying the following machine learning techniques, computers were able to accurately classify surgical maneuvers. Random forest predictions of surgical video correctly classified 74% of all video segments into suturing, tying, and transition states for a randomly selected test set. By comparison, decision tree analysis correctly classified 64% of video segments. Hidden markov model adjustments improved the random forest predictions to 79% for simple interrupted suturing on a subset of randomly selected participants. CONCLUSION(S): Random forest predictions aided by hidden markov modeling provided the best prediction of surgical maneuvers. Marker-less video motion tracking combined with machine learning can accurately predict surgical maneuvers from a continuous video record and may enable more efficient video review of surgical procedures. (Figure Preseted).