RT-DNAS: Real-Time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation

Abstract

Accurately segmenting temporal frames of cine magnetic resonance imaging (MRI) is a crucial step in various real-time MRI guided cardiac interventions. To achieve fast and accurate visual assistance, there are strict requirements on the maximum latency and minimum throughput of the segmentation framework. State-of-the-art neural networks on this task are mostly hand-crafted to satisfy these constraints while achieving high accuracy. On the other hand, existing literature has demonstrated the power of neural architecture search (NAS) in automatically identifying the best neural architectures for various medical applications, within which differentiable NAS is a prevailing and efficient approach. However, they are mostly guided by accuracy, sometimes with computation complexity, but the importance of real-time constraints are overlooked. A major challenge is that such constraints are non-differentiable and thus are not compatible with the widely used differentiable NAS frameworks. In this paper, we present a strategy that can directly handle real-time constraints in differentiable NAS frameworks, named RT-DNAS. Experiments on extended 2017 MICCAI ACDC dataset show that compared with state-of-the-art manually and automatically designed architectures, RT-DNAS is able to identify neural architectures that can achieve better accuracy while satisfying the real-time constraints.