A Deep Learning-Based Interactive Medical Image Segmentation Framework

Abstract

Image segmentation is an essential component in medical image analysis. The case of 3D images such as MRI is particularly challenging and time consuming. Interactive or semi-automatic methods are thus highly desirable. While deep learning outperforms classical methods in automatic segmentation, its use in interactive frameworks is still limited. The main reason is that most neural networks do not lend themselves well to the required user interaction loop. We propose a general deep learning-based interactive framework for image segmentation, which embeds a base network in a user interaction loop with a user feedback memory. We propose to model the memory explicitly as a sequence of consecutive framework states, from which the features can be learned. A major difficulty is related to training, as the network inputs include the user feedback and thus depend on the network’s previous output. We propose to introduce a virtual user in the training process, modelled by simulating the user feedback from the current segmentation. We demonstrate our framework on the task of female pelvis MRI segmentation, using a new dataset. We evaluate our framework against existing work with the standard metrics and conduct a user evaluation. Our framework outperforms existing systems.