Deep learning in real-time image-guided surgery: a systematic review of applications, methodologies, and clinical relevance

Abstract

Aim: Real-time image guidance using deep learning is being increasingly used in surgery. This systematic review aims to characterize intraoperative systems, mapping applications, performance and latency, validation practices, and the reported effects on workflow and patient-relevant outcomes. Methods: A systematic review was conducted on PubMed, Embase, Scopus, ScienceDirect, IEEE Xplore, Google Scholar, and Directory of Open Access Journals from December 31, 2024. Eligible English-language, peer-reviewed diagnostic accuracy, cohort, quasi-experimental, or randomized studies (2017-2024) evaluated the learning for real-time intraoperative guidance. Two reviewers screened, applied the Joanna Briggs Institute checklists, and extracted the design, modality, architecture, training, validation, performance, and latency. Heterogeneity precluded the meta-analysis. Results: Twenty-seven studies spanning laparoscopic, neurosurgical, breast, colorectal, cardiac, and other workflows met the criteria. The modalities included red-green-blue laparoscopy or endoscopy, ultrasound, optical coherence tomography, cone-beam computed tomography, and stimulated Raman histology. The architectures were mainly convolutional neural networks with frequent transfer learning. Reported performance was high, with classification accuracy commonly 90%-97% and segmentation Dice or intersection over union up to 0.95 at operating-room-compatible speeds of about 20-300 frames per second or sub-second per-frame latency; volumetric pipelines sometimes required up to 1 min. Several systems demonstrated intraoperative feasibility and high surgeon acceptance, yet fewer than one quarter reported external validation and only a small subset linked outputs to patient-important outcomes. Conclusion: Deep-learning systems for real-time image guidance exhibit strong technical performance and emerging workflow benefits. Priorities include multicenter prospective evaluations, standardized reporting of latency and external validation, rigorous human factors assessment, and open benchmarking to demonstrate generalizability and patient impact.