A Multi-task Network with Weight Decay Skip Connection Training for Anomaly Detection in Retinal Fundus Images

Abstract

By introducing the skip connection to bridge the semantic gap between encoder and decoder, U-shape architecture has been proven to be effective for recovering fine-grained details in dense prediction tasks. However, such a mechanism cannot be directly applied to reconstruction-based anomaly detection, since the skip connection might lead the model overfitting to an identity mapping between the input and output. In this paper, we propose a weight decay training strategy to progressively mute the skip connections of U-Net, which effectively adapts U-shape network to anomaly detection task. Thus, we are able to leverage the modeling capabilities of U-Net architecture, and meanwhile prevent the trained model from bypassing low-level features. Furthermore, we formulate an auxiliary task, namely histograms of oriented gradients (HOG) prediction, to encourage the framework to deeply exploit contextual information from fundus images. The HOG feature descriptors with three different resolutions are adopted as the auxiliary supervision signals. The multi-task framework is dedicated to enforce the model to aggregate shared significant commonalities and eventually improve the performance of anomaly detection. Experimental results on Indian Diabetic Retinopathy image Dataset (IDRiD) and Automatic Detection challenge on Age-related Macular degeneration dataset (ADAM) validate the superiority of our method for detecting abnormalities in retinal fundus images. The source code is available at https://github.com/WentianZhang-ML/WDMT-Net.