Improving endoscopic smoke detection with semi-supervised noisy student models

Abstract

Laparoscopic surgery consists of many tasks that have to be handled by the surgeon and the operating room personnel. Recognition of situations where action is required enables automatic handling by the integrated OR or notifying the surgical team with a visual reminder. As a byproduct of some surgical actions, electrosurgical smoke needs to be evacuated to keep the vision clear for the surgeon. Building on the success of convolutional neural networks (CNNs) for image classification, we utilize them for image based detection of surgical smoke. As a baseline we provide results for an image classifier trained on the publicly available smoke annotions of the Cholec80 dataset. We extend this evaluation with a self-training approach using teacher and student models. A teacher model is created with the labeled dataset and used to create pseudo labels. Multiple datasets with pseudo labels are then used to improve robustness and accuracy of a noisy student model. The experimental evaluation shows a performance benefit when utilizing increasing amounts of pseudo-labeled data. The state of the art with a classification accuracy of 0.71 can be improved to an accuracy of 0.85. Surgical data science often has to cope with minimal amounts of labeled data. This work proposes a method to utilize unlabeled data from the same domain. The good performance in standard metrics also shows the suitability for clinical use.