Simulation-based regularized logistic regression

Abstract

In this paper, we develop a simulation-based framework for regu-larized logistic regression, exploiting two novel results for scale mixtures of nor-mals. By carefully choosing a hierarchical model for the likelihood by one type of mixture, and implementing regularization with another, we obtain new MCMC schemes with varying e±ciency depending on the data type (binary v. binomial, say) and the desired estimator (maximum likelihood, maximum a posteriori, poste-rior mean). Advantages of our omnibus approach include oexibility, computational e±ciency, applicability in p À n settings, uncertainty estimates, variable selection, and assessing the optimal degree of regularization. We compare our methodology to modern alternatives on both synthetic and real data. An R package called reglogit is available on CRAN. © 2012 International Society for Bayesian Analysis.