Bayesian estimation and model averaging of convolutional neural networks by hypernetwork

Abstract

Neural networks have rich ability to learn complex representations. However, due to the limited number of training samples, overfitting is likely to occur. Hence, it is essential to regularize the learning process of neural networks. In this paper, we propose a regularization method which estimates CNN's parameters as probabilistic distributions by using hypernet. Then, to make it applicable to a large model such as WideResNet, we used likelihood as loss function. Experimental results demonstrate the regularization of our method.