A Joint 3D+2D Fully Convolutional Framework for Subcortical Segmentation

Abstract

In this paper, we proposed and validated a novel joint 3D+2D fully convolutional framework for segmenting subcortical structures from magnetic resonance images (MRIs). A 2D Attention U-net (AU-net) following a multi-atlas guided 3D fully convolutional network (MF-net) is constructed in the proposed framework. A novel multi-atlas based encoding block for learning both prior expert information and MRI intensity profile, and a novel attention block for learning structural boundary information are respectively proposed in the 3D MF-net and the 2D AU-net. In the joint 3D+2D framework, the to-be-segmented image and the 2D probability maps for each structure of interest (obtained from the 3D MF-net) were sliced into a 2D image set at each of the three orthogonal views (axial, sagittal, coronal) and then fed into three trained 2D AU-nets, which yields superior segmentation performance. Validation experiments were performed on two datasets respectively contain 16 and 18 T1-weighted MRIs. Compared to several existing state-of-the-art segmentation methods including a multi-atlas joint label fusion method and three representative fully convolutional network methods, the proposed method performed significantly better for a majority of the 12 subcortical structures, with the overall mean Dice scores being respective 0.917 and 0.865 for the two datasets.