Home Fitness and Rehabilitation Support System Implemented by Combining Deep Images and Machine Learning Using Unity Game Engine

Abstract

In this study, we aim to develop a game support system that allows users to work out and rehabilitate at home alone. The system first reads the user’s depth image through the Kinect v2 sensing interface, converts it into the user’s skeleton, then continues to track the human body and creates the required database through machine learning after recording the dynamic changes in the skeleton. This information is then applied to a game platform designed with the Unity game engine. Finally, the game screen is connected to smart glasses via Bluetooth, allowing users to experience the game in augmented reality (AR). The database is constructed via the adaptive enhanced AdaBoost algorithm used in machine learning, and the architecture of the Unity game platform is edited in C#. The support system of the home fitness and rehabilitation game is completed after being combined with Kinect v2. There are two modes in the game platform, fitness and rehabilitation, with five movements in the fitness mode and 10 movements in the rehabilitation mode. Both modes have three sub-modes: independent training, coach demonstration, and mini-games. We demonstrated through tests that the system can allow users to easily and comfortably rehabilitate and work out at home as if there were a coach guiding them. Therefore, in addition to effectively improving the accuracy of movements, the system can also help avoid injuries or accidents caused by inaccurate movements.