Iterative local hyperlinear learning based relief for feature weight estimation

Abstract

Feature weighting is considered as an important machine learning approach to deal with the problem of estimating the quality of attributes for pattern classification applications. Local Hyperlinear Learning based Relief (LH-Relief) was shown to be very efficient in estimating attributions in high-dimensional data involving irrelevant noises. However, the convergence of LH-Relief can not be guaranteed. In this paper, we propose an innovative feature weighting algorithm to solve the problem of LH-Relief, called Iterative Local Hyperlinear Learning based Relief (ILH-Relief). ILH-Relief is based on LH-Relief using classical margin maximization. The key idea is to estimate the feature weights through local approximation and gradient descent. To demonstrate the viability and the effectiveness of our formulation for feature selection in supervised learning, we perform extensive experiments on both UCI and Microarray datasets. The proposed algorithm can save at least half of feature ranking time with a better classification performance compared with other feature weighting methods.