Convolutional Neural Network with Stacked Autoencoder for Kernel Initialization

Abstract

In recent years, convolutional neural networks have gained popularity in the area of image processing, machine translation, speech recognition, object detection, and many other tasks. The data generated in all these areas are very large, and there are a large number of samples along with a large number of attributes. Areas like bioinformatics have a large amount of data, but face the problem of a small number of samples with a large number of attributes. In all these applications, initialization plays a better role for better generalization of the network. In this work, we have proposed a novel approach for kernel initialization in which the weights learned by each autoencoder hidden layer acts as the initial kernel (filter) weight of each convolutional neural network layer. The result of the proposed approach is compared with random initialization of the kernel weights. The results show that the proposed method performs comparably to random initialization.