Exploiting LSTM for Joint Object and Semantic Part Detection

Abstract

Object detection and semantic part detection are two tasks that can mutually benefit each other. Thus, in this paper we propose an approach to perform joint detection. Our approach is built upon a proposal-driven detection framework. In order to explore the mutual interaction between two tasks, we integrate an interaction module into the detection framework. The module contains a relationship modeling stage and a feature fusion stage. The former determines the relationship between each object proposal and semantic part proposal by formulating it as a binary classification problem. The second stage adopts Long Short Term Memory networks (LSTMs) to fuse the features of an object and its associated parts, as well as the features of a part and its associated object proposals. Experiments on publicly available datasets show that, with the proposed interaction module, our joint detection approach consistently outperforms the baselines using object or part appearance only. Our approach also shows superior performance when compared to other semantic part detection methods.