Bayesian partially-protected regularization as a model selection tool

Abstract

This work first describes Bayesian Partially-Protected Lasso (BPL), which combines the power of Bayesian Lasso with the ability to protect key theoretical explanatory variables from shrinkage to a zero effect in the model. This approach allows researchers to identify protected and non-protected variables so that data with many explanatory variables can be efficiently machine-explored without sacrificing theoretically important predictors. We provide the statistical background, algorithms, examples, and easy to use tools in an R package. We then introduce BPEN–Bayesian Protected Elastic Net estimation process that builds on the idea of the Bayesian Partially-Protected Lasso. Since the Elastic Net adds a second penalty term to the standard Lasso it provides a more flexible regularization process. This is a novel approach that combines the robustness of the Elastic Net in sifting through potentially large sets of variables while simultaneously safeguarding the integrity of those grounded in theoretical principles.