Overview of Big Data Analytics in Modern Astronomy

Abstract

Astronomers are increasingly compelled to chart the universe with ever greater precision. Projects like the Sloan Digital Sky Survey (SDSS), Pan-STARRS, and the Large Synoptic Survey Telescope (LSST) generate approximately 100-200 Petabytes of data annually, presenting significant big data challenges in terms of storage, processing, and data transfer. The Square Kilometer Array (SKA), an ambitious project involving 130,000 antennas and 200 dishes spanning two continents, is scheduled to become operational in 2028. It will collect 160 terabytes of data per second, translating to 1 petabyte of data daily. Coping with this immense volume of data necessitates real-time processing and analysis, driving the need for efficient machine learning and image analysis algorithms. Astronomy stands as an ideal domain for big data analytics, pushing the boundaries of data analysis. This review paper will present intriguing applications for data scientists, exploring the challenges and recent technological advancements in big data analytics concerning astronomy. The paper will also critically assess the strengths and weaknesses of various approaches, methodologies, or tools used in big data analytics within the context of astronomy, supported by relevant case studies.