Improving the learning speed in 2-Layered LSTM network by estimating the configuration of hidden units and optimizing weights initialization

Abstract

This paper describes a method to initialize the LSTM network weights and estimate the configuration of hidden units in order to improve training time for function approximation tasks. The motivation of this method is based on the behavior of the hidden units and the complexity of the function to be approximated. The results obtained for 1-D and 2-D functions show that the proposed methodology improves the network performance, stabilizing the training phase. © Springer-Verlag Berlin Heidelberg 2008.