Online learning: Searching for the best forgetting strategy under concept drift

Abstract

Learning from data streams in the presence of concept drifts has become an important application area.When the environment changes, it is necessary to rely on on-line learning with the capability to forget out-dated information. Ensemble methods have been among the most successful approaches because they do not need hard-coded and difficult to obtain prior knowledge about the changes in the environment. However, the management of the committee of experts which ultimately controls how past data is forgotten has not been thoroughly investigated so far. This paper shows the importance of the forgetting strategy by comparing several approaches. The results lead us to propose a new ensemble method which compares favorably with the well-known CDC system based on the classical "replace the worst experts" forgetting strategy. © Springer-Verlag 2013.