Automatic Detection of Camera Rotation Moments in Trans-Nasal Minimally Invasive Surgery Using Machine Learning Algorithm

Abstract

Background: Minimally invasive surgery (MIS) is an advanced surgical technique that relies on a camera to provide the surgeon with a visual field. When the camera rotates along its longitudinal axis, the horizon of the surgical view tilts, increasing the difficulty of the procedure and the cognitive load on the surgeon. To address this, we proposed training a convolutional neural network (CNN) to detect camera rotation, laying the groundwork for the automatic correction of this issue during MIS procedures. Methods: We collected trans-nasal MIS procedure videos from YouTube and labeled each frame as either “tilted” or “non-tilted”. The dataset consisted of 2116 video frames, with 497 frames labeled as “tilted” and 1619 frames as “non-tilted”. This dataset was randomly divided into three subsets: training (70%), validation (20%), and testing (10%) Results: The ResNet50 was trained on the dataset for 10 epochs, achieving an accuracy of 96.9% at epoch 6 with a validation loss of 0.0242 before validation accuracy began to decrease. On the test set, the model achieved an accuracy of 96% with an average loss of 0.0256. The final F1 score was 0.94, and the Matthews Correlation Coefficient was 0.9168, with no significant bias toward either class. The trained ResNet50 model demonstrated a high success rate in predicting significant camera rotation without favoring the more frequent class in the dataset. Conclusions: The trained CNN accurately detected camera rotation with high precision, establishing a foundation for developing an automatic correction system for camera rotation in MIS procedures.