Neural Network Approach for 2-Dimension Person Pose Estimation with Encoded Mask and Keypoint Detection

Abstract

We present an innovative approach for 2D person pose estimation by developing a convolutional neural network for human 2-channel mask prediction and human 2D pose estimation. Conceptually our idea is simple, inspired by prior image segmentation research in general. We establish a perception that explicitly encoded mask data can be served as a critical feature for person pose estimation. We propose a convolution neural network model by combining the image segmentation technique with the bottom-up approach for human pose estimation. We observe that the construction of a two stage-network for training in an end-to-end manner is beneficial to one another: for person mask prediction and 2D person pose estimation. At the pose estimation stage, we detect heat-maps against the person keypoints location from the mask information and their mutual connection relations. They are then used to estimate an ultimate pose in a way to remove the unwanted or occluded keypoints, as those keypoints may propagate across the network and lead to redundant pose estimation. We train and test our system on the MS-COCO dataset, and the experimental results validate the superior efficiency of the proposed methodology.