Analysis of The Influence Activation Function, Learning Rate And Momentum in Determining Mean Square Error (MSE) in Restricted Boltzmann Machines (RBM) Neural Network

Abstract

Restricted Boltzmann Machines (RBM) is a neural network unsupervised learning algorithm which only consists of two layers of the visible layer and the hidden layer. RBM performance is strongly influenced by parameters such as the activation function that is used to activate neurons in the network and the learning rate and momentum to accelerate the learning process. Selection of the activation function corectly very influence the performance in determining the Mean Square Error (MSE) on RBM neural network. Activation function that is used on RBM network is the sigmoid activation function. Several variants of the sigmoid activation function like binary sigmoid and sigmoid hyperbolic tangent (tanh). By using datasets MNIST for learning and testing, it appears that the success rate for the classification of the binary sigmoid activation function, is determined by the value of MSE is small. Unlike the tangent activation function MSE ascending of rising number of epoch. Activation function binary sigmoid with the learning rate of 0.05 and momentum 0.7 has a recognition rate of handwriting a high namely 93.42%, followed by the learning rate 0.01 momentum 0.9 namely 91.92%, for learning rate 0.05 momentum 0.5 namely 91.31%, learning rate 0.01 and momentum 0.7 is 90.56% and the last learning rate 0.01 and momentum 0.5 namely 87.49%