The comparison of performance according to initialization methods of deep neural network for malware dataset

Abstract

Training Deep NN, proper initialization of weights leads to good performance. The methods that commonly used to initialize the weights are, limited variance of weight’s values and re-use unsupervised pre-trained weights. In this paper, we proposed the new algorithm that some of weights are used after being pre-trained by the CD method of unsupervised DBN and the other of the weights are initialized using the Xavier or He initialization method. We call these DBNnX and DBNnHe. We compare the performance with several DBNnX, DBNnHe and existing methods. We evaluated and visualized these by using AUC score and using box plot. As the result of experiment, we found the DBN2X and DBN2He are best.