From Artificial Neural Networks to Deep Learning: A Research Survey

Abstract

Deep learning (DL) is one of the hot topics in the field of artificial intelligence. As an extension of artificial neural network (ANN), deep learning models have more powerful learning and adaptation capabilities and they develop more and more rapidly. At present, because of the great advantages compared with the traditional methods, the development of convolutional neural network (CNN) is the hottest and it is widely used. This article first introduces the development of DL in chronological order and then analyses and summarizes artificial neural networks, and then uses the example of the LeNet-5 to explain the basic network structure of CNN and describe its training methods comprehensively. Based on the CNN model, many new convolutional neural networks with improved structure are analysed, such as AlexNet, ZFNet, VGGNet, GoogleNet and so on. The principle of each method is introduced in detail and the accuracy is compared with each other to conclude the correlation between the depth and the accuracy of DL. Then, for each method, it discusses some current problems existing in the corresponding method of DL, and points out future research and application directions worth exploring of each method. Finally, it summarizes the history of DL and the application of specific methods and the existing problems and solutions.