Evolutionary structure minimization of deep neural networks for motion sensor data

Abstract

Many Deep Neural Networks (DNNs) are implemented with the single objective to achieve high classification scores. However, there can be additional objectives like the minimization of computational costs. This is especially important in the field of mobile computing where not only the computational power itself is a limiting factor but also each computation consumes energy affecting the battery life. Unfortunately, the determination of minimal structures is not straightforward. In our paper, we present a new approach to determine DNNs employing reduced structures. The networks are determined by an Evolutionary Algorithm (EA). After the DNN is trained, the EA starts to remove neurons from the network. Thereby, the fitness function of the EA is depending on the accuracy of the DNN. Thus, the EA is able to control the influence of each individual neuron. We introduce our new approach in detail. Thereby, we employ motion data recorded by accelerometer and gyroscope sensors of a mobile device. The data are recorded while drawing Japanese characters in the air in a learning context. The experimental results show that our approach is capable to determine reduced networks with similar performance to the original ones. Additionally, we show that the reduction can improve the accuracy of a network. We analyze the reduction in detail. Further, we present arising structures of the reduced networks.