A sequential dynamic multi-class model and recursive filtering by variational Bayesian methods

Abstract

Adaptive classification evolving over time is an important learning task that arises in many applications. In this paper, a sequential dynamic multi-class model (SDMM) is proposed for representing the multi-class adaptive learning task, which is based on the polychotomous response model and dynamic logistic regression. Multiple state chains in the SDMM are coupled by the observable labels and feature vectors. Each state chain is modeled as a first-order Markov process with time-varying covariance parameters for characterizing the non-stationary generating process of sequential labels. Augmented auxiliary variables are introduced for developing efficient inference procedures according to the popular data augmentation strategy. Variational Bayesian methods are applied to estimate the dynamic state variables and augmented auxiliary variables recursively. According to the results of recursive filtering procedures using mean-field approximation forms, one-step-ahead predicted probabilities are calculated by marginalizing the state variables. Experiment results based on both synthetic and real data show that the proposed model significantly outperforms the non-sequential static methods for the multi-class adaptive learning problems with missing labels. Encouraging results have been obtained by comprising well-known multi-class data stream algorithms. © 2011 Springer-Verlag.