Feature selection for hidden markov models with discrete features

Abstract

Hidden Markov models (HMMs) are widely used for modeling multivariate time series data. However, all collected data is not always useful for distinguishing between states. In these situations, feature selection should be implemented to save the expense of collecting and processing low utility data. Feature selection for HMMs has been studied but most existing methods assume that the observed data follows a Gaussian distribution. In this paper, a method for simultaneously estimating parameters and selecting features for an HMM with discrete observations is presented. The presented method is an extension of the feature saliency HMM which was originally developed to incorporate feature cost into the feature selection process. Expectation-maximization algorithms are derived for features following a Poisson distribution and features following a discrete non-parametric distribution. The algorithms are evaluated on synthetic data sets and a real-world event detection data set that is composed of both Poisson and non-parametric features.