Inclusion of Pre-Processing and Time Series Algorithms in Map Reduce Environment using Big Data Analytics

Abstract

Map Reduce is one of the most effective ways of handling Big Data. Many of existing Data Mining / AI algorithms was developed in Map Reduce to provide effective results. There are many more algorithms including preprocessing algorithms such as Binarization, Normalization etc., Time series algorithms such as Moving average, Sliding Window, Correlation etc., which are not yet implemented in Map Reduce. Although there are not major algorithms they play a vital role in preprocessing and processing chunk data to a meaningful data. In this paper, we proposed a model of including these algorithms in Map Reduce to improve preprocessing outcome of Big Data much faster. The processed data can then be trained by the regression algorithms using Machine learning techniques to preprocess the huge data in a long run automatically.