Network intrusion detection system by learning jointly from tabular and text-based features

Abstract

Network intrusion detection systems (NIDS) play a critical role in maintaining the security and integrity of computer networks. These systems are designed to detect and respond to anomalous activities that may indicate malicious intent or unauthorized access. The need for robust NIDS solutions has never been more pressing in today's digital landscape, characterized by constantly evolving cyber threats. Deploying effective NIDS can be challenging, particularly in accurately identifying network anomalies amid the ever-increasing sophisticated and difficult-to-detect cyber threats. The motivation for our research stems from the recognition that while NIDS studies have made significant strides, there remains a crucial need for more effective and accurate methods to detect network anomalies. Commonly used features in NIDS studies include network logs, with some studies exploring text-based features such as payload. However, traditional machine and deep learning models may need to be improved in learning jointly from tabular and text-based features. Here, we present a new approach that integrates both tabular and text-based features to improve the performance of NIDS. Our research aims to address the existing limitations of NIDS and contribute to the development of more reliable and efficient network security solutions by introducing more effective and accurate methods for detecting network anomalies. Our internal experiments have revealed that the deep learning approach utilizing tabular features produces favourable results, whereas the pre-trained transformer approach needs to perform sufficiently. Hence, our proposed approach, which integrates both feature types using deep learning and pre-trained transformer approaches, achieves superior performance. These findings indicate that integrating both feature types using deep learning and pre-trained transformer approaches can significantly improve the accuracy of network anomaly detection. Moreover, our proposed approach outperforms the state-of-the-art methods in terms of accuracy, F1-score, and recall on commonly used NIDS datasets consisting of ISCX-IDS2012, UNSW-NB15, and CIC-IDS2017, with F1-scores of 99.80%, 92.37%, and 99.69%, respectively, indicating its effectiveness in detecting network anomalies.