Ecosystem Description of Hadoop Platform Based on HDFS, MapReduce and Data Warehouse Tool Hive

Abstract

This paper introduces the processing process of the distributed file system (HDFS, MapReduce) which is the core of the Hadoop distributed computing platform and introduces the data warehouse tool Hive and the distributed database Hbase. Spark is a big data distributed programming framework, which not only implements MapReduce operator map function and reduce function and calculation model, but also provides more abundant operators. This paper describes the ecosystem of Hadoop platform based on HDFS, MapReduce and data warehouse tool Hive.