Feature Selection from Local Lift Dependence-Based Partitions

Abstract

The classical approach to feature selection consists in minimizing a cost function of the estimated joint distribution of the variable of interest and the feature vectors. However, in order to estimate the joint distribution, and therefore, the cost function, it is necessary to discretize the variables, so that feature selection algorithms are partition dependent, as they depend on the partitions in which the variables are discretized. In this framework, this paper aims to propose a systematic approach to the discretization of random vectors, which is based on the Local Lift Dependence. Our approach allows an interpretation of the local dependence between the variable of interest and the selected features, so that it is possible to outline the kind of dependence that exists between them. The proposed approach is applied to study the dependence between the performances on entrance exam subjects and on first semester courses of University of São Paulo Statistics and Computer Science undergraduate programs.