Enhancing semantic features with compositional analysis for scene recognition

Abstract

Scene recognition systems are generally based on features that represent the image semantics by modeling the content depicted in a given image. In this paper we propose a framework for scene recognition that goes beyond the mere visual content analysis by exploiting a new cue for categorization: the image composition, namely its photographic style and layout. We extract information about the image composition by storing the values of affective, aesthetic and artistic features in a compositional vector. We verify the discriminative power of our compositional vector for scene categorization by using it for the classification of images from various, diverse, large scale scene understanding datasets. We then combine the compositional features with traditional semantic features in a complete scene recognition framework. Results show that, due to the complementarity of compositional and semantic features, scene categorization systems indeed benefit from the incorporation of descriptors representing the image photographic layout (+ 13-15% over semantic-only categorization). © 2012 Springer-Verlag.