Tests of Statistical Methods for Estimating Galaxy Luminosity Function and Applications to the Hubble Deep Field

Abstract

We study statistical methods for the estimation of the luminosity function (LF) of galaxies. We focus on four nonparametric estimators: the 1/Vmax estimator, the stepwise maximum-likelihood estimator, Chołoniewski's estimator, and the improved Lynden-Bell estimator. The performance of the 1/Vmax estimator has recently been questioned, especially for the faint-end estimation of the LF. We improve these estimators for studies of the distant universe, and examine their performances for various classes of functional forms by Monte Carlo simulations. We also apply these estimation methods to the mock Two-Degree Field (2dF) redshift survey catalog prepared by Cole et al. We find that the 1/Vmax estimator yields a completely unbiased result if there is no inhomogeneity, but is not robust against clusters or voids. This is consistent with the well-known results, and we do not confirm the bias trend of 1/Vmax claimed by Willmer in the case of a homogeneous sample. We also find that the other three maximum-likelihood type estimators are quite robust and give consistent results with each other. In practice, we recommend Chołoniewski's estimator for two reasons: (1) it simultaneously provides the shape and normalization of the LF; and (2) it is the fastest among these four estimators, because of its algorithmic simplicity. We then analyze the photometric redshift data of the Hubble Deep Field prepared by Fernández-Soto et al. using the above four methods. We also derive the luminosity density, ρL, at the B and I bands. Our B-band estimation is roughly consistent with that of Sawicki, Lin, & Yee, but it is a few times lower at 2.0