A Novel Video Anomaly Detection Method Based on Multi-scale Swin Transformer and Memory-augmented Attention Feature Fusion

Abstract

The purpose of Video Anomaly Detection is to automatically identify abnormal spatiotemporal patterns in surveillance systems. Traditional autoencoder frame reconstruction methods have made great progress in abnormal video detection. However, these methods often fail to adequately correlate global features with local emphasis feature and to harness feature information at various scales within the network. With the aim of addressing these issues, this study proposes a novel multi-scale Swin Transformer and memory-augmented attention feature fusion network for video anomaly detection(MSTMAF), which effectively performs multi-scale cross-fusion of global and fine-grained local features of the autoencoder. Specifically, a multi-scale Swin Transformer encoder is designed to achieve multi-layer feature fusion while extracting video frame features through a multi-branch skip connection operation. Furthermore, we design multi-attention fusion mechanism to refine attention, reducing the influence of background location in the feature map. This process heightens the high-level semantic representation of local features. Additionally, we introduce memory augmented module to better retain archetypal features of normal behavior from historical data. MSTMAF-Net achieved outstanding AUC performance on four publicly available datasets: UCSD Ped1(0.881), UCSD Ped2(0.982), CUHK Avenue (0.913), ShanghaiTech (0.764) and UCF-crime(0.831) demonstrating the effectiveness of our study.