Packet marking based cooperative attack response service for effectively handling suspicious traffic

Abstract

The security vulnerabilities in a network environment and their corresponding countermeasures have become more critical issues than ever. Although many researchers and vendors have introduced powerful mechanisms such as Intrusion Detection System (IDS) or Intrusion Prevention System (IPS) for network security, the packet-based decision is not always correct, especially when those systems are involved in network traffics across multiple organizations under different security policies. In fact, some legitimate (normal) network traffics produce a similar pattern to that of malicious traffics such as Distributed Denial of Service (DDoS), and vice versa. We call those traffics suspicious. Suspicious traffic cannot be clearly designated as malicious or normal traffic. Since traditional IDS or IPS approaches make a simple binary decision (i.e., allow or reject) based on pre-defmed rules, there is a high possibility that suspicious/legitimate packets are rejected or suspicious/malicious packets are allowed. To enhance the quality of service in a network environment, we propose in this paper a Packet Marking-Based Cooperative Attack Response Service (pm-CARS) that is able to effectively deal with suspicious network traffic. pm-CARS nodes cooperate with each other by using packet-marking. These pm-CARS nodes mark suspicious packets instead of dropping them. All the marked packets are forwarded to the next node using a low priority of service designation, which indicates the drop probability is very high. Our pm-CARS includes two schemes: abnormal IP address detection and abnormal excess traffic detection schemes. Our pm-CARS can reduce the false-positive rate and can protect the quality of service for innocent traffic from attacks. Finally, we simulate our ideas in a network environment and discuss the evaluation results. © Springer-Verlag Berlin Heidelberg 2006.