Intelligent Hybrid Learning Framework for Adaptive Real-Time Cyber Threat Detection and Response

Abstract

With the rapid evolution of cyberattacks, conventional security mechanisms struggle to detect and respond to threats in real time. This work presents an intelligent hybrid learning framework that integrates supervised machine learning with deep reinforcement learning to achieve adaptive, explainable, and efficient cybersecurity. The model employs an XGBoost classifier trained on the UNSW-NB15 dataset for initial threat prediction and a Deep Q-Network (DQN) agent, implemented using PyTorch, to autonomously determine optimal defense actions against evolving attack patterns. The DQN interacts with live or simulated network data, refining its policy dynamically through continuous reward-based feedback. Essential network attributes such as protocol type, port numbers, packet size, and inter-arrival time are standardized for accurate and uniform analysis. A rules-based labeling engine complements the dataset where public data is limited. The system features a Flask-based dashboard for real-time monitoring, SHAP-driven explainability, and detailed performance visualization. Delivering high detection accuracy and adaptability, this framework unites the precision of machine learning with the adaptability of reinforcement learning—forming a transparent, future-ready cybersecurity model capable of intelligent and continuous threat management in real-time environments.