AQUAdex: A highly efficient indexing and retrieving method for astronomical big data of time series images

Abstract

In the era of Big Data, scientific research is challenged with handling massive data sets. To actually take advantage of Big Data, the key problem is to retrieve the desired cup of data from the ocean, as most applications only need a fraction of the entire data set. As the indexing and retrieving method is intrinsically connected with specific features of the data set and the goal of research, a universal solution is hardly possible. Designed for efficiently querying Big Data in astronomy time domain research, AQUAdex, a new spatial indexing and retrieving method is proposed to extract Time Series Images form Astronomical Big Data. By mapping images to tiles (pixels) on the celestial sphere, AQUAdex can complete queries 9 times faster, which is proven by theoretical analysis and experimental results. AQUAdex is especially suitable for Big Data applications because of its excellent scalability. The query time only increases 59 % while the data size grows 14 times larger.