Scalable and Generalizable Cross-Domain Invariant Network Analysis and Intrusion Detection System Using Deep Learning

Abstract

Modern networks are increasingly complex and diverse - especially with the rapid growth of the Internet of Things (IoT) and Industrial IoT (IIoT) - so intrusion detection (ID) models that are tightly coupled to a single dataset or environment often fail when deployed elsewhere. This work studies cross-domain generalization within network-traffic domains and proposes a scalable neural classifier trained with quasi-Newton optimization: NN-BFGS in WEKA (MLPClassifier) and its low-memory variant NN-LBFGS in Python (scikit-learn). We evaluate on three widely used corpora: CICIDS-2017 (general network traffic), MQTT-IDS (protocol-specific), and ToN-IoT (system/host telemetry). Across MQTT scenarios, the proposed model achieves > 99.90% accuracy with near-perfect recall and minimal false positives; on CICIDS-2017 it is robust across seven attack groups. We also assess scalability on merged scenarios, where performance is retained. From a feature perspective, we identify a compact set of 3 feature types (packet transmission, transmission pattern, and flags) that are invariant between the network-traffic datasets and can guide future model design and dataset construction. Finally, we present a reusable workflow architecture to facilitate building similar scalable ID systems.