Leveraging Knowledge Graphs and Deep Learning for automatic art analysis

Abstract

The growing availability of large collections of digitized artworks has disclosed new opportunities to develop intelligent systems for the automatic analysis of fine arts. Among other benefits, these tools can foster a deeper understanding of fine arts, ultimately supporting the spread of culture. However, most of the systems proposed in the literature are only based on visual features of digitized artwork images, which are sometimes only integrated with some metadata and textual comments. A Knowledge Graph (KG) that integrates a rich body of information about artworks, artists, painting schools, etc., in a unified structured framework, can provide a valuable resource for more powerful information retrieval and knowledge discovery tools in the artistic domain. To this end, in this paper we present ArtGraph:1 an artistic KG based on WikiArt and DBpedia. The graph already provides knowledge discovery capabilities without having to train a learning system. In addition, we propose a novel KG-enabled fine art classification method based on ArtGraph, which is used to perform artwork attribute prediction tasks. The method extracts embeddings from ArtGraph and injects them as “contextual” knowledge into a Deep Learning model. Compared to the state-of-the-art, the proposed model provides encouraging results, suggesting that the exploitation of KGs in combination with Deep Learning can pave the way for bridging the gap between the Humanities and Computer Science communities.