Detecting TCP SYN Flood Attack in the Cloud

Abstract

In this paper, an approach to protecting virtual machines (VMs) against TCP SYN flood attack in a cloud environment is proposed. An open source cloud platform Eucalyptus is deployed and experimentation is carried out on this setup. We investigate attacks emanating from one VM to another in a multi-tenancy cloud environment. Various scenarios of the attack are executed on a webserver VM. To detect such attacks from a cloud provider's perspective, a security mechanism involving a packet sniffer, feature extraction process, a classifier and an alerting component is proposed and implemented. We experiment with k-nearest neighbor and artificial neural network for classification of the attack. The dataset obtained from the attacks on the webserver VM is passed through the classifiers. The artificial neural network produced a F1 score of 1 with the test cases implying a 100% detection accuracy of the malicious attack traffic from legitimate traffic. The proposed security mechanism shows promising results in detecting TCP SYN flood attack behaviors in the cloud.