A Review of the Overfitting Problem in Convolution Neural Network and Remedy Approaches

Abstract

Deep learning methods have attracted much attention over the past few years after their breakthroughs in speech recognition and computer vision. However, Convolution Neural Network (CNN) is one of the most important networks that have been used especially in image classification, these networks are facing an essential problem which is the “overfitting problem”. This problem means that the difficulty implies that the model would just "memorize" previously observed patterns, rather than “learn” the important patterns. As a result, it will down the classification performance and become a huge problem. Different remedy approaches have been suggested, each one can exhibit different behavior in the reduction of the overfitting problem according to the nature of the training data. So, this variety of remedy approaches needs to be examined. This paper would be focused on the factors that can cause the overfitting problem, then continue into the approaches to solving this problem.