Uncertainty estimation for strong-noise data

Abstract

The measurement of uncertainty in classification tasks is a challenging problem. Bayesian neural networks offer a standard mathematical framework to address the issue but limited by the high computational cost. Recently, several non-Bayesian approaches like Deep Ensemble are proposed as alternatives. However, most of the works focus on measuring the model uncertainty rather than the uncertainty over the data. In this paper, we demonstrate that noise in the training data has an adverse impact on uncertainty estimation, and we prove that Deep Ensemble is ineffective when training on the strong-noise dataset. We propose an easy-implemented model to estimate the uncertainty on noisy datasets, which is compatible with many existing classification models. We test our method on Fashion MNIST, Fashion MNIST with different levels of Gaussian noise, and the strong-noise financial dataset. The experiments show that our approach is effective on each dataset, whether it contains strong noise or not. The usage of our method improves the trading strategy to increase the annual profit by nearly 5%.