Clinically-relevant Summarisation of Cataract Surgery Videos Using Deep Learning

Abstract

Cataract surgery is one of the most frequently performed medical procedures worldwide, an estimated 20 million such surgeries occurring annually. However, the training required to become a competent cataract surgeon takes years due to its challenging technical nature. This limits the supply of capable surgeons. One aspect of modern cataract surgery is that video recordings are routinely taken using microscope cameras, and these recordings can be used to review errors and improve technique throughout surgical training. However, reviewing raw surgery video footage is tedious and may not lead to actionable insights improving surgeon performance. To tackle this issue, a novel artificial intelligence (AI)-based framework for the extraction of detailed surgery video summary statistics directly from the raw surgery footage is proposed. The input to the system is a video of a cataract surgery procedures and the output is a summary report. The approach uses deep learning models (ResNet-50, ResNet-152 and InceptionV3 were tested) to identify and time surgical instrument activity. Additionally, a unique dataset consisting of 57,422 hand-labelled frames extracted from a new locally-sourced video dataset of 29 retrospective cataract surgery recordings was created. Testing these predictive models with 4-fold cross validation across ten different surgical instruments resulted in a best mean testing prediction area under the ROC curve of 97.6%, and a mean testing sensitivity of 96.6%. Given these high levels of accuracy, the reports generated by our system are high quality and could be used to provide actionable insight into surgical technique during surgical training.