A metric learning approach for endoscopic kidney stone identification

Abstract

Several Deep Learning (DL) methods have recently been proposed for an automated identification of kidney stones during an ureteroscopy to enable rapid therapeutic decisions. Even if these DL approaches led to promising results, they are mainly appropriate for kidney stone types for which numerous labeled data are available. However, only few labeled images are available for some rare kidney stone types. This contribution exploits Deep Metric Learning (DML) methods (i) able to generate low-dimensionality data representations, and (ii) leading to a satisfactory kidney stone identification performance with a small amount of endoscopic images. The proposed Guided Deep Metric Learning approach is based on a novel architecture which was designed to learn enhanced data representations. The solution was inspired by Few-Shot Learning (FSL) and makes use of a teacher-student approach. The teacher model (GEMINI) generates a reduced hypothesis space from the labeled data, which is then used as prior knowledge to “guide” a student model (i.e., ResNet50) through a Knowledge Distillation scheme. Extensive tests were first performed on two datasets separately used for the recognition, namely a set of images acquired for the surfaces of the kidney stone fragments, and a set of images of the fragment sections. The proposed DML-approach improved the identification accuracy by 10% and 12% in comparison to DL-methods and other DML-approaches, respectively. Moreover, model embeddings from the two dataset types were merged in an organized way through a multi-view scheme to simultaneously exploit the information of surface and section fragments. Test with the resulting mixed model improves the identification accuracy by at least 3% and up to 30% with respect to DL-models and shallow machine learning methods, respectively.