Choice of Hyperparameter Values for Convolutional Neural Networks Based on the Analysis of Intra-network Processes

Abstract

One of the critical tasks that have to be solved when forming a convolutional neural network (CNN) is the choice of the values of its hyperparameters. Existing attempts to solve this problem are based, as a rule, on one of two approaches. The first of them implements a series of experiments with different values of the hyperparameters of CNN. For each of the obtained sets of hyperparameter values, training is carried out for the corresponding network versions. These experiments are performed until we obtain a CNN with acceptable characteristics. This approach is simple to implement but does not guarantee high performance for CNN. In the second approach, the choice of network hyperparameter values is treated as an optimization problem. With the successful solution of such a problem, it is possible to obtain a CNN with sufficiently high characteristics. However, this task has considerable complexity, and also requires a large consumption of computing resources. This article proposes an alternative approach to solving the problem of choosing the values of hyperparameters for CNN, based on an analysis of the processes taking place in the network. We demonstrate the efficiency of this approach by solving the problem of classifying functional dependencies as an example.