Data reduction and noise filtering for predicting times series

Abstract

In this paper we introduce a modification of the real discrete Fourier transform and its inverse transform to filter noise and perform reduction on the data whilst preserving the trend of global moving of time series. The transformed data is still in the same time domain as the original data, and can therefore be directly used by any other mining algorithms. We also present a classification algorithm MinCov in this paper. Given a new data tuple, it provides values for each class that measures the likelihood of the tuple belonging to that class. The experimental results show that the MinCov algorithm is comparable to C4.5, and using MinCov as a mining algorithm the average hit rate of predicting the sign of stock return is 23.92% higher than that on the original data. This means that the predicting accuracy has been remarkably improved by means of the proposed data reduction and noise filtering method.