Regional Gating Neural Networks for Multi-label Image Classification

Abstract

This paper proposes a novel deep learning framework for multi-label image classification, namely regional gating neural networks (RGNN). The motivation is two folds. First, global image features (including CNN based features) ignore the underlying context information among different objects in an image. Consequently, people attempt to use information from objectness regions. However, current objectness region proposal algorithms usually produce several thousand region candidates, including many classification irrelevant or even noisy regions. This leads to the second problem: how to select useful contextual regions for image classification. RGNN is an end-to-end deep learning framework that can automatically select contextual region features with specially designed gate units, which are then fused for classification. Because the gate units and the classifier are integrated in the same deep neural network pipeline, we can learn parameters of the network simultaneously. We evaluate the proposed method on PASCAL VOC 2007/2012 and MS-COCO benchmarks, and results show that RGNN is superior to existing state-of-the-art methods.