Sparse gradient pursuit for robust visual analysis

Abstract

Many high-dimensional data analysis problems, such as clustering and classification, usually involve the minimization of a Laplacian regularization, which is equivalent to minimize square errors of the gradient on a graph, i.e., the disparity among the adjacent nodes in a data graph. However, the Laplacian criterion usually preserves the locally homogeneous data structure but suppresses the discrimination among samples across clusters, which accordingly leads to undesirable confusion among similar observations belonging to different clusters. In this paper, we propose a novel criterion, named Sparse Gradient Pursuit (SGP), to simultaneously preserve the within-class homogeneity and the between-class discrimination for unsupervised data clustering. In addition, we show that the proposed SGP criterion is generic and can be extended to handle semi-supervised learning problems by incorporating the label information into the data graph. Though this unified semi-supervised learning model leads to a nonconvex optimization problem, we develop a new numerical scheme for the SGP related nonconvex optimization problem and analyze the convergence property of the proposed algorithm under mild conditions. Extensive experiments demonstrate that the proposed algorithm performs favorably against the state-of-the-art unsupervised and semi-supervised methods.