An active and dynamic botnet detection approach to track hidden concept drift

Abstract

Nowadays, machine learning has been widely used as a core component in botnet detection systems. However, the assumption of machine learning algorithm is that the underlying botnet data distribution is stable for training and testing, which is vulnerable to well-crafted concept drift attacks, such as mimicry attacks, gradient descent attacks, poisoning attacks and so on. In this paper we present an active and dynamic learning approach to mitigate botnet hidden concept drift attacks. Instead of passively waiting for false negative, this approach could actively find the trend of hidden concept drift attacks using statistical p-values before performance starts to degenerate. And besides periodically retraining, this approach could dynamically reweight predictive features to track the trend of underlying concept drift. We test this approach on the public CTU botnet captures provided by malware capture facility project. The experiment results show that this approach could actively get insights of botnet hidden concept drift, and dynamically evolve to avoid model aging.