Effective probability forecasting for time series data using standard machine learning techniques

Abstract

This study investigates the effectiveness of probability forecasts output by standard machine learning techniques (Neural Network, C4.5, K-Nearest Neighbours, Naive Bayes, SVM and HMM) when tested on time series datasets from various problem domains. Raw data was converted into a pattern classification problem using a sliding window approach, and the respective target prediction was set as some discretised future value in the time series sequence. Experiments were conducted in the online learning setting to model the way in which time series data is presented. The performance of each learner's probability forecasts was assessed using ROC curves, square loss, classification accuracy and Empirical Reliability Curves (ERC) [1]. Our results demonstrate that effective probability forecasts can be generated on time series data and we discuss the practical implications of this. © Springer-Verlag Berlin Heidelberg 2005.