Proposing Posture Recognition System Combining MobilenetV2 and LSTM for Medical Surveillance

Abstract

This paper proposes a posture recognition system that can be applied for medical surveillance. The proposed method estimates human posture using mobilenetV2 and long short-term memory (LSTM) to extract the important features of an image. The output of the system was a fully estimated skeleton. We used seven human indoor postures, including lying, sitting, crouching, standing, walking, fighting, and falling, and classified them. The output results are the extraction of the human skeleton and the corresponding labels for the poses. We first experiment with classification using machine learning. The system only achieves approximately 88% accuracy because it is not able to classify similar postures, such as standing and walking. This difference can be caused by the extraction of features for static images, and the machine learning classification algorithm has not reached accuracy with training data. Therefore, we proposed the integration of the LSTM model into the proposed system. LSTM learns the features of the skeleton and provides classification results for postures. As a result, our system improved the accuracy by up to 99%. Similar postures, such as standing and walking, have improved accuracy by up to 7%. In addition, we performed the system on the Jetson Nano hardware. The results show that it can run on a low-profile (44% CPU and 2.1 frames per second) that is capable of applications for remote patient monitoring devices.