Robust Loss Function for Deep Learning Regression with Outliers

Abstract

In regression analysis, the presence of outliers in the data set can strongly distort the classical least squares (known as “L2”) estimator and lead to unreliable results (due to the large abnormal error registered by outliers compared to the error of the majority of the training samples). To deal with this, several robust-to-outliers methods in robust statistics have been proposed in the statistical literature. However, in the context of deep regression networks, very few efforts have been carried out to deal with outliers and most deep regression algorithms make use of the traditional L2 loss function. In this paper, we consider the issue of training deep neural networks in the context of robust regression. As such, we introduce a robust deep regression model which is based on a novel robust loss function. With this latter, our model is able to adapt to an outlier distribution, without requiring any hard threshold on the proportion of outliers in the training set. Experimental evaluations on a head pose estimation dataset show that our model generalizes well to noisy datasets, compared to other state-of-the-art techniques.