Bayesian density regression with logistic Gaussian process and subspace projection

Abstract

We develop a novel Bayesian density regression model based on logistic Gaussian processes and subspace projection. Logistic Gaussian processes provide an attractive alternative to the popular stick-breaking processes for modeling a family of conditional densities that vary smoothly in the conditioning variable. Subspace projection offers dimension reduction of predictors through multiple lin-ear combinations, offering an alternative to the zeroing out theme of variable selec-tion. We illustrate that logistic Gaussian processes and subspace projection com-bine well to produce a computationally tractable and theoretically sound density regression procedure that offers good out of sample prediction, accurate estima-tion of subspace projection and satisfactory estimation of subspace dimensionality. We also demonstrate that subspace projection may lead to better prediction than variable selection when predictors are well chosen and possibly dependent on each other, each having a moderate influence on the response. © 2010 International Society for Bayesian Analysis.