Top-κ supervise feature selection via admm for integer programming

Abstract

Recently, structured sparsity-inducing based feature selection has become a hot topic in machine learning and pattern recognition. Most of the sparsity-inducing feature selection methods are designed to rank all features by certain criterion and then select the κ top-ranked features, where κ is an integer. However, the κ top features are usually not the top κ features and therefore maybe a suboptimal result. In this paper, we propose a novel supervised feature selection method to directly identify the top κ features. The new method is formulated as a classic regularized least squares regression model with two groups of variables. The problem with respect to one group of the variables turn out to be a 0-1 integer programming, which had been considered very hard to solve. To address this, we utilize an efficient optimization method to solve the integer programming, which first replaces the discrete 0-1 constraints with two continuous constraints and then utilizes the alternating direction method of multipliers to optimize the equivalent problem. The obtained result is the top subset with κ features under the proposed criterion rather than the subset of κ top features. Experiments have been conducted on benchmark data sets to show the effectiveness of proposed method.