LightNets: The Concept of Weakening Layers

Abstract

Deep neural networks generally use the information fusion at the front and back layers because the traditional convolutional networks that stack convolutional layers have limited ability to extract the effective information or effective information cannot be passed to the back layer. These networks, which incorporate the information of the previous layer, are attributed to the improvement of the accuracy achieved to facilitate the propagation of gradients and incorporate various techniques for adjusting parameters. We have explored the relationship between the residual networks and dense networks and found that they have a great degree of similarity in certain situations and can combine them to exert their respective advantages. We propose a lightweight network (LightNets) architecture with few parameters. LightNets has faster training and inference speeds because it has fewer parameters. The main way to explore is to change the flow and integration of information. This paper compares various network architectures and regularization techniques to analyze the generalization capabilities of the LightNets.