Optimization Methods for Convolutional Neural Networks – The LeNet-5 Algorithm

Abstract

Convolutional neural networks are enhanced version of fully connected neural networks. The neural networks are used to recognize objects after training the neural network system for some datasets that can also be divided into classes at the output. These networks were a breakthrough in computer vision filed for object recognition where the system can optimize its parameters for better results with using feed forward and back propagation. The convolutional neural networks reduced the time of training and testing the dataset by replacing the full network nodes connecting to each node in the subsequent layer to some nodes or filter to each subsequent layer node. There are many algorithms for convolutional neural networks ranging from simple algorithms to complex ones. Each algorithm has different hidden layers with different hyper parameters and filters. The activation functions and number of nodes in each layer for each algorithm may be different. The applications for these convolutional neural networks cover many fields such as hand written digit recognition, alphabet handwritten recognition, and any group of objects that can be divided into classes such as cloth, X-ray imaging and many more. The LeNet-5 algorithm is one of the convolutional neural networks. With full analysis of this algorithm, I will prove that a simple module of the algorithm can provide maximum accuracy and minimum loss function than the original algorithm.