Combining deep learning and ASP-based models for the semantic segmentation of medical images

Abstract

Automatic segmentation represents a huge breakthrough in computer aided diagnosis and medicine, thanks to its ability in extracting and providing important information useful to clinicians for interventional and diagnostic tasks. Several approaches based on Deep Learning (DL), such as Convolutional Neural Networks, have been proposed to identify anatomical and pathological structures, and to extract hidden patterns from huge amounts of data; they already proved to be suitable for learning without human supervision; however, drive the decisions of the network according to prior knowledge is still a challenging issue, as well as interpreting the choices made by the models. In this work we propose the use of deductive rule-based approaches, such as Answer Set Programming (ASP) to drive DL approaches in performing semantic segmentation of medical images. Specifically, we encoded some available prior medical knowledge via ASP, thus defining a rule-based model for deducting all admitted combinations of classes and right locations in medical images. We make use of the inferred knowledge to: (i)define a novel loss function which includes penalty for misclassified elements identified by the network, and(ii) perform post-processing to remove small islands of noise and identified classes which do not comply with medical requirements; also, we re-assign misclassified elements to the more frequent class in the neighborhood. We evaluated our approach using different artificial neural networks trained for automatic semantic segmentation based on Laryngeal Endoscopic Images; the resulting framework relies on ASP programs to include structural and medical properties in DL-based techniques, paving the way to an effective combination of deductive and inductive methods.