Advanced Technique for Imbalance Mitigation in Predictive Monitoring and Anomaly Detection System

Abstract

This paper presents an advanced approach to improving network traffic monitoring systems using machine learning algorithms. The main attention is paid to the problems of data imbalance and insufficient labeling in real communication systems. These problems often lead to inaccurate anomaly detection and unreliable system performance. To solve these problems, the paper proposes a dynamic class weighting technique that improves anomaly detection, especially when dealing with uncertain or unevenly represented data. The technique ensures that minority classes, such as malicious or anomalous traffic, are properly accounted for during model training, which improves overall detection accuracy. This approach provides the ability to dynamically change class weights based on new input data, and the simplicity of the model, because it is linear and does not have many layers, allows for relatively quick retraining. In addition, the paper describes an optimized data preparation process that facilitates efficient training of neural networks. These networks are integrated into proactive monitoring modules, which allows for real-time detection of network anomalies and potential threats. Although the proposed multiclass approach yields slightly lower global metrics (Precision 0.88, Recall 0.88) than the binary baseline, it significantly improves malicious traffic detection by introducing an additional class for uncertain samples, thus offering a more realistic and robust representation of network behavior. This proactive approach is particularly useful in today's communications environments, which are characterized by increasing traffic volumes and greater data diversity. By providing rapid detection and response to network breaches, the proposed solution increases the reliability and stability of networks, providing more robust protection against new cyber threats. The approach is particularly well suited for dynamic and complex networks, where traditional static monitoring methods often prove insufficient. The techniques presented in this article thus contribute to the development of more intelligent and responsive network monitoring systems that can cope with the complexities of modern communication infrastructures, where the demand for real-time analysis and anomaly detection continues to grow.