A comparison of ensemble and case-base maintenance techniques for handling concept drift in spam filtering

Abstract

The problem of concept drift has recently received considerable attention in machine learning research. One important practical problem where concept drift needs to be addressed is spam filtering. The literature on concept drift shows that among the most promising approaches are ensembles and a variety of techniques for ensemble construction has been proposed. In this paper we compare the ensemble approach to an alternative lazy learning approach to concept drift whereby a single case-based classifier for spam filtering keeps itself up-to-date through a case-base maintenance protocol. The case-base maintenance approach offers a more straightforward strategy for handling concept drift than updating ensembles with new classifiers. We present an evaluation that shows that the case-base maintenance approach is at least as effective as a selection of ensemble techniques. The evaluation is complicated by the overriding importance of False Positives (FPs) in spam filtering. The ensemble approaches can have very good performance on FPs because it is possible to bias an ensemble more strongly away from FPs than it is to bias the single classifer. However this comes at considerable cost to the overall accuracy. Copyright © 2006, American Association for Artificial Intelligence (www.aaai.org). All rights reserved.