Combined Deep Learning of Fundus Images and Fluorescein Angiography for Retinal Artery/Vein Classification

Abstract

We present a new framework for achieving highly accurate retinal artery/vein classification based on a combined input of fundus images and corresponding fluorescein angiography (FA) images. Although the observable bloodflow in FA sequences should be a definitive cue to discern artery and vein, it is often insufficient due to limitations in frame rate and image quality. Since fundus images are acquired by default, we incorporate both the fundus image and FA as the input to a deep learning framework comprising convolutional and graph neural networks. First, a convolutional neural network with a parallel structure is learned to extract features from the two image modalities. Then, these features are used as input to dual hierarchical graph neural networks for pixelwise classification based on the long-range connectivity of vessels. We provide empirical evidence based on ablative and comparative quantitative evaluations supporting the effectiveness of the proposed network configuration in combining the fundus image and FA. We also provide comparative evaluation supporting the improvement in performance compared to previous methods.