Graph Database Solution for Higher-order Spatial Statistics in the Era of Big Data

Abstract

We present an algorithm for the fast computation of the general N -point spatial correlation functions of any discrete point set embedded within an Euclidean space of . Utilizing the concepts of kd-trees and graph databases, we describe how to count all possible N -tuples in binned configurations within a given length scale, e.g., all pairs of points or all triplets of points with side lengths < r MAX . Through benchmarking, we show the computational advantage of our new graph-based algorithm over more traditional methods. We show measurements of the three-point correlation function up to scales of ∼200 Mpc (beyond the baryon acoustic oscillation scale in physical units) using current Sloan Digital Sky Survey (SDSS) data. Finally, we present a preliminary exploration of the small-scale four-point correlation function of 568,776 SDSS Constant (stellar) Mass (CMASS) galaxies in the northern Galactic cap over the redshift range of 0.43 <  z  < 0.7. We present the publicly available code GRAMSCI (GRAph Made Statistics for Cosmological Information; bitbucket.org/csabiu/gramsci ), under a Gnu is Not Unix (GNU) General Public License.