An End-to-End Human Segmentation by Region Proposed Fully Convolutional Network

Abstract

Person segmentation in images has various applications, for example, smart home, human-computer interaction, and scene perception for self-driving cars, which are a key feature of the Internet of Things. Due to limitations in performance, such as accuracy and runtime, most traditional methods do not fulfill the practical requirements. Deep learning-based modern segmentation systems become prevalent. Fully convolutional network (FCN), as a classic image semantic segmentation method, directly optimizes the semantic map from the original image in a pixel-wise manner without using pixel-correlations or global object information. In this paper, we propose an efficient end-to-end person segmentation network structure fusing the person detection network with the FCN. The person detection network estimates the region of interest of persons and enforces the segmentation network to focus on the optimization of person segmentation. The loss function of the proposed network considers both the segmentation error and the detection bias error. In addition, the lightweight design of the detection network that optimizes only person bounding-box coordinates enables real-time person detection. The experimental comparison and analysis of several different networks on several datasets show the effectiveness of the proposed fusion strategy. The approach shows a promising practical application potential by fast running time and high segmentation accuracy.