A Survey on Big Data Management and Job Scheduling

Abstract

Big data has gained its popularity in the recent years due to the fact that there is a need for sophisticated method to collect, process, analyze and visualize huge volumes of data generated by our digital and computing world. Several challenges in handling petabytes of information, commonly named as Big data needs to be addressed in more efficient way. Big data management (BDM) is the process of collecting, storing, analysing and visualization of large volumes of data, which can be in the form of structured, unstructured and semistructured formats. Problems such as data acquisition, data storage, data retrieval, data analysis, and data visualization can no longer be handled by traditional database systems. The primary purpose of this paper is to provide a comprehensive survey on Big data management and to provide an overview on various algorithms related to job scheduling in Hadoop and the latest advancements. These research directions can lead to exploration of Big data domain and result in development of optimal techniques and scheduling algorithms to address problems faced in Big data.