Pedestrian Activity Recognition from 3D Skeleton Data using Long Short Term Memory Units

Abstract

The pace of advancement in the realm of autonomous driving is quickening, raising concerns and escalating expectations for pedestrian safety, intelligence, and stability. In dynamic and uncertain contexts, some scenarios necessitate distinguishing pedestrian position and behavior, such as crossing or standing. The ability to recognize a pedestrian is a critical component of autonomous driving success. Before making an appropriate response, the vehicle must detect the pedestrian, identify their body movements, and comprehend the significance of their actions. In this paper, a detailed description of the architecture for 3D activity recognition of a pedestrian using Recurrent Neural Networks (RNN) is presented. In this work, a custom dataset that was created from an autonomous vehicle of RRLAB at the Technische Universität Kaiserslautern is employed. The information was gathered for behaviors such as parallel crossing, perpendicular crossing, texting, and phone calls, among others. On the data, models were trained, and Long-Short Term Memory (LSTM), a recurrent neural network has shown to be superior to Convolution Neural Networks (CNN) in terms of accuracy. Various investigations and analyses have revealed that two models trained independently for upper and lower body joints produced better outcomes than one trained for all joints. On a test data, it had a 97 percent accuracy for lower body activities and an 88-90 percent accuracy for upper body activities.