An end-to-end multi-task system of automatic lesion detection and anatomical localization in whole-body bone scintigraphy by deep learning

Abstract

Limited by spatial resolution and visual contrast, bone scintigraphy interpretation is susceptible to subjective factors, which considerably affects the accuracy and repeatability of lesion detection and anatomical localization. In this work, we design and implement an end-to-end multi-task deep learning model to perform automatic lesion detection and anatomical localization in whole-body bone scintigraphy. A total of 617 whole-body bone scintigraphy cases including anterior and posterior views were retrospectively analyzed. The proposed semi-supervised model consists of two task flows. The first one, the lesion segmentation flow, received image patches and was trained in a supervised way. The other one, skeleton segmentation flow, was trained on as few as five labeled images in conjunction with the multi-atlas approach, in a semi-supervised way. The two flows joint in their encoder layers so each flow can capture more generalized distribution of the sample space and extract more abstract deep features. The experimental results show that the architecture achieved the highest precision in the finest bone segmentation task in both anterior and posterior images of whole-body scintigraphy. Such an end-to-end approach with very few manual annotation requirement would be suitable for algorithm deployment. Moreover, the proposed approach reliably balances unsupervised labels construction and supervised learning, providing useful insight for weakly labeled image analysis.