The effect of optimizers on CNN architectures for art style classification

Abstract

The artistic style of a painting is one of the most frequent semantic criteria used to classify paintings. However, identifying the unique style of a painting is a complex task, and usually only art experts do it, as it requires significant knowledge and expertise. Thus, it is required to employ the advances of deep learning approaches in image processing to present automatic methods to the art community to do such tasks as an enormous number of digital paintings are available on the internet. In this study, we propose a framework to compare the performances of six pre-Trained convolutional neural networks (Xception, ResNet50, InceptionV3, InceptionResNetV2, DenseNet121, and EfficientNet B3) for identifying the artistic style of a painting, including Xception architecture, which to our knowledge has never been used for this purpose before. Furthermore, we study the effect of three different optimizers such as (SGD, RMSprop, and Adam) with two learning rates (1e-2 and 1e-4) on the performance of the models using transfer learning to find the best hyper-parameters for each model. Our experiments using two art classification datasets, Pandora18k and Painting-91, indicated that InceptionResNetV2 is the most accurate model for style classification on both datasets when it was trained with an Adam optimizer and a learning rate equal to 1e-4.