A novel metric for the evaluation of IDSs effectiveness

Abstract

Nowadays intrusion detection system (IDS) has a considerable attention as a crucial element in network security. The question that arises is which IDS is effective for our system? The answer should inevitably take into account the evaluation of IDSs effectiveness. Dealing with this challenge, many valuable evaluation metrics have been introduced such as receiver operating characteristic (ROC) curve, Bayesian detection rate, intrusion detection capability, intrusion detection operating characteristic, cost-based metrics, etc. The benefits and drawbacks of these metrics are discussed in this paper. We subsequently propose a novel metric called intrusion detection effectiveness (EID) that manipulates the drawbacks of the existing ones, taking into account all essential and related parameters. We demonstrate the utility of EID over the previously proposed ones, and how it realizes the measurement of the actual effectiveness rather than the relative effectiveness as followed by the existing ones. EID can be used for evaluating the wired or wireless IDSs effectiveness. Additionally, we conduct experimental evaluation of two popular wireless IDSs (WIDSs), Kismet and AirSnare, to illustrate the benefits of EID.