Title:Tests of Statistical Methods for Estimating Galaxy Luminosity Function and Applications to the Hubble Deep Field

Abstract: We studied the statistical methods for the estimation of the luminosity function (LF) of galaxies. We focused on four nonparametric estimators: $1/V_{\rm max}$ estimator, maximum-likelihood estimator of Efstathiou et al. (1988), Chołoniewski's estimator, and improved Lynden-Bell's estimator. The performance of the $1/V_{\rm max}$ estimator has been recently questioned, especially for the faint-end estimation of the LF. We improved these estimators for the studies of the distant Universe, and examined their performances for various classes of functional forms by Monte Carlo simulations. We also applied these estimation methods to the mock 2dF redshift survey catalog prepared by Cole et al. (1998). We found that $1/V_{\rm max}$ estimator yields a completely unbiased result if there is no inhomogeneity, but is not robust against clusters or voids. This is consistent with the well-known results, and we did not confirm the bias trend of $1/V_{\rm max}$ estimator claimed by Willmer (1997) in the case of homogeneous sample. We also found that the other three maximum-likelihood type estimators are quite robust and give consistent results with each other. In practice we recommend Chołoniewski's estimator for two reasons: 1. it simultaneously provides the shape and normalization of the LF; 2. it is the fastest among these four estimators, because of the algorithmic simplicity. Then, we analyzed the photometric redshift data of the Hubble Deep Field prepared by Fernández-Soto et al. (1999) using the above four methods. We also derived luminosity density $\rho_{\rm L}$ at $B$- and $I$-band. Our $B$-band estimation is roughly consistent with that of Sawicki, Lin, & Yee (1997), but a few times lower at $2.0 < z < 3.0$. The evolution of $\rho_{\rm L}(I)$ is found to be less prominent.