An Analysis on Very Deep Convolutional Neural Networks: Problems and Solutions

  • Pricope T
N/ACitations
Citations of this article
5Readers
Mendeley users who have this article in their library.

Abstract

Neural Networks have become a powerful tool in computer vision because of the recent breakthroughs in computation time and model architecture. Very deep models allow for better deciphering of the hidden patterns in the data; however, training them successfully is not a trivial problem, because of the notorious vanishing/exploding gradient problem. We illustrate this problem on VGG models, with 8 and 38 hidden layers, on the CIFAR100 image dataset, where we visualize how the gradients evolve during training. We explore known solutions to this problem like Batch Normalization (BatchNorm) or Residual Networks (ResNets), explaining the theory behind them. Our experiments show that the deeper model suffers from the vanishing gradient problem, but BatchNorm and ResNets do solve it. The employed solutions slighly improve the performance of shallower models as well, yet, the fixed deeper models outperform them.

Cite

CITATION STYLE

APA

Pricope, T.-V. (2021). An Analysis on Very Deep Convolutional Neural Networks: Problems and Solutions. Studia Universitatis Babeș-Bolyai Informatica, 66(1), 5. https://doi.org/10.24193/subbi.2021.1.01

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free