RGB-D scene recognition based on object-scene relation and semantics-preserving attention

Abstract

Scene recognition is challenging due to intra-class diversity and inter-class similarity. Previous works recognize scenes either with global representations or with intermediate representations of objects. By contrast, we investigate more discriminative sequential representation of object-to-scene relations (SOSRs) for scene recognition. Particularly, we develop an Attention-Preserving Memory-Learning (APML) model, which enforces the Memory Network of the semantic domain to guide the Learning Network of the appearance domain in the learning procedure. Accordingly, we allocate semantics-preserving attention to different objects, which is more effective to seek the key encoded SOSR and discard the misleading encoded SOSR between objects and scene without requiring extra labeled data. Based on the proposed APML networks, we obtain the state-of-the-art results of RGB-D scene recognition on SUN RGB-D and NYUD2 datasets.