An Empirical Study of Neural Network Hyperparameters

Abstract

The learning algorithms related to deep learning involves many attributes called hyperparameters, these variables help in determining the network structure. The performance of algorithms depends upon these hyper-parameter variables that are needed to be set prior to the actual implementation of the algorithm. This study involves an overview of some of the commonly used hyperparameters in the context of learning algorithms used for training neural networks along with the analysis of adaptive learning algorithms used for tuning learning rates.