A comparative study of the lasso-type and heuristic model selection methods

Abstract

This study presents a first comparative analysis of Lasso-type (Lasso, adaptive Lasso, elastic net) and heuristic subset selection methods. Although the Lasso has shown success in many situations, it has some limitations. In particular, inconsistent results are obtained for pairwise highly correlated predictors. An alternative to the Lasso is constituted by model selection based on information criteria (IC), which remain consistent in the situation mentioned. However, these criteria are hard to optimize due to a discrete search space. To overcome this problem, an optimization heuristic (Genetic Algorithm) is applied. To this end, results of aMonte-Carlo simulation study together with an application to an actual empirical problem are reported to illustrate the performance of the methods.