Learning-based misalignment detection for 2-D/3-D overlays

Abstract

In minimally invasive procedures, a standard routine of observing the operational site is using image guidance. X-ray fluoroscopy using C-arm systems is widely used. In complex cases, overlays of preoperative 3-D images are necessary to show structures that are not visible in the 2-D X-ray images. The alignment quality may degenerate during an intervention, e. g. due to patient motion, and a new registration needs to be performed. However, a decrease in alignment quality is not always obvious, as the clinician often focuses on structures which are not visible in the 2-D image, and only these structures are visualized in the overlay. In this paper, we propose a learning-based method for detecting different degrees of misalignment. The method is based on point-to-plane correspondences and a pre-trained neural network originally used for detecting good correspondences. The network is extended by a classification branch to detect different levels of misalignment. Compared to simply using the normalized gradient correlation similarity measure as a basis for the decision, we show a highly improved performance, e. g. improving the AUC score from 0.918 to 0.993 for detecting misalignment above 5mm of mean re-projection distance.