ABDviaMSIFAT: Abnormal Crowd Behavior Detection Utilizing a Multi-Source Information Fusion Technique

Abstract

Detecting abnormal crowd behavior in surveillance videos is a significant challenge due to the intricate and constantly evolving crowd dynamics. To solve this issue, we suggest a new method that combines data from various sources with different characteristics to enhance the precision of detecting human behavior in crowds. Our approach involves two separate pipelines that work simultaneously to produce scores for frames in a video segment. These scores are later modified for the individual level of the group, allowing for behavior recognition through the assessment of fuzzy logic functions. In the first pipeline, we utilize a depth-wise Separable Convolutional Neural Network (DWS-CNN) that provides reduced filtering compared to standard CNNs. The second pipeline combines the LiteFlowNet detector with the MOSSE tracker and a DSC-GRU network to generate high-level captions for objects in video frames. We implement the weighted average (WA) method to improve anomaly detection accuracy. Methods like weighted averages can mitigate the influence of outliers and noise in the outcomes or evaluations. Utilizing linguistic variables to represent scores and computing weighted averages of scores from two pipelines enhances the quality and reliability of these variables, creating fuzzy predicates that characterize people’s movements, presence, and responses at a microscopic scale. Our approach exceeds conventional visual and motion-centric methods, enabling a more comprehensive grasp of abnormal behaviors. Our suggested approach outperforms state-of-the-art methods in terms of effectiveness and performance based on tests done on well-known datasets.