Big Data Processing-Beyond Batch Processing

Abstract

This paper mainly focus on analysis of large sets of students data with one of the batch processing analysis techniques Beyond batch process, analysis of data streaming is done based on program of word counting program which executes data with HDFS along with dynamic created data. To compute similar coherent strategies one can implement a schema named batch and streaming process which dynamically creates data. The architecture is reduced to serve as X-Platform which uses ample number of tools for batch and stream analysis on this proposed frame work. Here we use spark-sql, a query language which acts as interface for interactive process to have iterative processes. Real time streaming data processing involves spark streaming works. Here we focus on preliminary evaluation of results and analysis report which compares data sets performance and also achieve low latency rate with usage of RDD.