Automatic Feedback For Physiotherapy Exercises Based On PoseNet

Abstract

This project helps patients who undergo physiotherapy exercises. It can help patients to do right poses and prevent them getting injured during the exercises. The home-based physiotherapy exercise (HPTE) dataset is used. The dataset consists of eight exercises which focus on shoulder, and knee. It is performed by five different actors with different shapes and sex. Our system uses seven exercises with two sides for more variety and better user experience. The main idea of the system is based on detecting the body key points using one of pose estimation techniques. The key points are extracted from different exercises and build our pickle file. Then, those pickle file's key points are compared to the key points of the patient using dynamic time warping. The system is implemented as a desktop application with a simple user interface and a 3D avatar is used to guide the patients to perform the exercises correctly. The system gives them a score on the exercise and a feedback on which parts of their body have incorrectly posture. Our system offers high accuracy pose estimation and pose comparison with a minimum accuracy of 80%.