Anomaly Detection Based on Vision Transformer Model and Texture Features

Abstract

Anomaly detection is one of the video surveillance applications, which aims to detect and analyze abnormal behaviors and risky situations in order to prevent accidents. Various deep learning models have been previously developed for this purpose, such as CNN, RNN, and Vision Transformer, each one has its strengths and weaknesses based on the quality of input data. This paper proposes a novel approach based on the texture characteristics of input frames. In order to enrich the input data of the vision transformer model, and enhance feature extraction for the detection of anomaly, we combine the original image with its texture extracted using Local Binary Pattern(LBP), and fed them into a fine-tuned pre-trained Vision Transformer, enabling the automatic classification of video frames into abnormal and normal categories. The results demonstrate the effectiveness of our approach in identifying risky situations in video sequences.