MDFULog: Multi-Feature Deep Fusion of Unstable Log Anomaly Detection Model

Abstract

Effective log anomaly detection can help operators locate and solve problems quickly, ensure the rapid recovery of the system, and reduce economic losses. However, recent log anomaly detection studies have shown some drawbacks, such as concept drift, noise problems, and fuzzy feature relation extraction, which cause data instability and abnormal misjudgment, leading to significant performance degradation. This paper proposes a multi-feature deep fusion of an unstable log anomaly detection model (MDFULog) for the above problems. The MDFULog model uses a novel log resolution method to eliminate the dynamic interference caused by noise. This paper proposes a feature enhancement mechanism that fully uses the correlation between semantic information, time information, and sequence features to detect various types of log exceptions. The introduced semantic feature extraction model based on Bert preserves the semantics of log messages and maps them to log vectors, effectively eliminating worker randomness and noise injection caused by log template updates. An Informer anomaly detection classification model is proposed to extract practical information from a global perspective and predict outliers quickly and accurately. Experiments were conducted on HDFS, OpenStack, and unstable datasets, showing that the anomaly detection method in this paper performs significantly better than available algorithms.