Target: Limbs movement guidance for learning physical activities with a video see-through head-mounted display

Abstract

In the aging society, people are paying more attention to having good exercise habits. The advancement of technology grants the possibility of learning various kinds of exercises using multi-media equipment, for example, watching instruction videos. However, it is difficult for users to learn accurate movements due to lack of feedback information. To explore what augmented reality (AR) or virtual reality (VR) technologies can assist the users, lots of research on movement guidance and evaluation has been purposed and developed. Chua et al. [Chua et al. 2003] built a Tai Chi Chuan training system in virtual reality environment. They created a virtual coach in front of the user. Integrated with motion capture system, users can see their movement in VR and compare it to the coach's movement. However, these systems can only let the users see their avatars instead of themselves. LightGuide [Sodhi et al. 2012] used a projector hanging from the ceiling and projected visual information on users' hands to guide hand movements. It could only guide users in a fixed space with their hands being under the projection zone. OutsideMe [Yan et al. 2015] used Kinect to capture the skeleton, RGB and depth images of users. They enabled users to see their body movements as external observers through a video see-through head-mounted display (VST-HMD). AR-Arm [Han et al. 2016] showed semitransparent arms to indicate the correct movement of Tai Chi Chuan. Users could follow the virtual arms intuitively to achieve accurate arm movement. In this paper, we present a full-body movement guidance system for learning physical activities with a VST-HMD. It contains a method for skeleton calibration and two interfaces for movement guidance: Coach-Surrounding Guidance and Ball-Following Guidance. We conducted a user study to evaluate the system on posture and movement learning.