Analysis of Big Data in Healthcare and Life Sciences Using Hive and Spark

Abstract

Big data is a declaration used to recognize the database whose area is afar the potential of typical database software tools to store, organize and examine. Big data has shown a new path toward the mankind. With several theoretical and technological obstacles in health huge processing, it is onerous to transfer knowledge into fortunate and valuable applications. Meeting the challenge of handling big data in healthcare information construction procedure, this paper proposes a referential architecture on the Hive and Spark platform to overcome the problems in healthcare big data process. Hive is a noteworthy project as a result of it permits exposing the simplest components of Hadoop, specifically map reduce and knowledge storage. Spark may be a memory-based computing framework that features a higher ability of computing and fault tolerance, supports batch, interactive, iterative and flow calculations. Experiment results of data upload, data query and data analysis show that the performance of the proposed framework is greatly improved, and a brief summary of the performance and the differences between two methods of Hive and Spark is also discussed.