Multi-person Pose Estimation in Complex Physical Interactions

Abstract

Recent literature addressed the monocular multi-person 3D human pose estimation task very satisfactorily. In these studies, different persons are usually treated as independent pose instances to estimate. However, in many every-day situations, people are interacting, and different pose instances should be considered jointly since the pose of an individual depends on the pose of his/her interactees. This work aims to develop machine learning techniques for human pose estimation of persons involved in complex interactions, using the interaction information to improve the performance. In this article, we will first describe the global problem, introduce the 3 main challenges and the related works. Then we will introduce the methods, experiments and results obtained with the person interaction network, PI-Net, which is accepted by IEEE WACV 2021. In this work we input the initial pose along with its interactees into a recurrent network to refine the pose of the person-of-interest, and we demonstrate the effectiveness of our method in the MuPoTS dataset, setting the new state-of-the-art on it. Finally, our ongoing works on constructing the person interaction dataset, PI dataset, and other future challenges will be discussed.