Human crime activity recognition and shooting weapon detection in video frames using the contour approximation algorithm, and FastDTW classifier

Abstract

Video surveillance analysis can assist in identifying several forms of criminal behaviors, including abuse, burglary, altercations, robbery, shootings, theft, and vandalism. The primary objective of this study is to identify human criminal acts, including kicking, dragging, beating, and shooting with firearms, along with their corresponding postures. This study utilized the human landmarks LEFT_WRIST, RIGHT_WRIST, LEFT_HIP, RIGHT_HIP, LEFT_ANKLE, and RIGHT_ANKLE extracted by the MediaPipe framework, to analyze human postures. The Euclidean distances from D1 to D10 among the landmark vectors of video frames postures (training vectors) and template postures (test vectors) served as the input for the Fast Dynamic Time Warping (DTW) classifier. The classifier evaluates the similarity between training and test vectors for Human Activity Recognition (HAR) and categorizes the crime. The Contour Approximation Algorithm (CAA) quantifies the Compressed Freeman Chain Code (CFCC) from CFCC1 to CFCC7 for both the human activity recognized Region of Interest (ROI), encompassing the LEFT_WRIST and RIGHT_WRIST landmarks and the ROI of the weapon template. The FastDTW classifier evaluates the similarity (CFCC1 to CFCC7) between the HAR ROI (training vectors) and the weapon template ROI (test vectors) to categorize the weapon. The empirical analysis revealed that the accuracies for HAR on the UCF-101 and Surveillance Fight datasets are 98.6% and 98.50% for small data samples, and 96.01% and 96.66% for large data samples, respectively. For WD on the same datasets, the accuracies are 99.18% and 98.11% for small data samples, and 97.22% and 95.89% for large data samples, respectively.