Penalized Maximum Likelihood

Abstract

Nonparametric density estimation methods are designed to estimate the density function f that generates a sample of observations, without assuming a parametric form for f. Rather, f is only assumed to be smooth, where smoothness is defined as the condition that at least a specified number of derivatives are square integrable. Penalized maximum likelihood estimators accomplish this by maximizing a penalized form of the log-likelihood, where the penalty discourages roughness in the final estimate. The estimator has a Bayesian interpretation, with the penalty function corresponding to the logged prior for the density. The resultant estimator often takes the form of a polynomial spline with knots at the order statistics, and asymptotically is approximately a local-bandwidth kernel estimator. The method can be adapted to large sparse contingency tables, resulting in cell probability estimators that are consistent under asymptotics that approximate such tables.