A Hybrid Neural Network for Graph-Based Human Pose Estimation from 2D Images

Abstract

This paper investigates the problem of human pose estimation (HPE) from single 2-dimensional (2D) still images using a convolutional neural network (CNN). The aim was to train the CNN to analyze a 2D input image of a person to determine the person's pose. The CNN output was given in the form of a tree-structured graph of interconnected nodes representing 2D image coordinates of the person's body joints. A new data-driven tree-based model for HPE was validated and compared to the traditional anatomy-based tree-based structures. The effect of the number of nodes in anatomy-based tree-based structures on the accuracy of HPE was examined. The tree-based techniques were compared with non-tree-based methods using a common HPE framework and a benchmark dataset. As a result of this investigation, a new hybrid two-stage approach to the HPE estimation was proposed. In the first stage, a non-tree-based network was used to generate approximate results that were then passed for further refinement to the second, tree-based stage. Experimental results showed that both of the proposed methods, the data-driven tree-based model (TD_26) and the hybrid model (H_26_2B) lead to very similar results, obtaining 1% higher HPE accuracy compared to the benchmark anatomy-based model (TA_26) and 3% higher accuracy compared to the non-tree-based benchmark (NT_26_A). The best overall HPE results were obtained using the anatomy-based benchmark with the number of nodes increased from 26 to 50, which also significantly increased the computational cost.