Dimensionality reduction and filtering on time series sensor streams

Abstract

This chapter surveys fundamental tools for dimensionality reduction and filtering of time series streams, illustrating what it takes to apply them efficiently and effectively to numerous problems. In particular, we show how least-squares based techniques (auto-regression and principal component analysis) can be successfully used to discover correlations both across streams, as well as across time. We also broadly overview work in the area of pattern discovery on time series streams, with applications in pattern discovery, dimensionality reduction, compression, forecasting, and anomaly detection. We aim to provide a unified view of time series stream mining techniques for dimensionality reduction (analysis and data reduction across streams) and filtering (analysis and data reduction across time). We describe methods that capture correlations and find hidden variables that describe trends in collections of streams. Discovered trends can then be used to quickly spot potential anomalies and do efficient forecasting. We describe a method which can incrementally find these correlation patterns and hidden variables, which summarize the key trends in the entire stream collection, with no buffering of stream values and without directly comparing pairs of streams. Moreover, it is any-time and dynamically detects changes. We also describe efficient online methods for quick forecasting (estimation of future values) and imputation (estimation of past, missing values) on multiple time series streams. Finally, we describe methods that can capture and summarize auto-correlations (correlations within a single series, across time), that also describe key trends. We also briefly explain how these techniques relate to others, and illustrate various trade-offs that are available to practitioners.