Robust Classification using Structured Sparse Representation Ehsan Elhamifar Ren�� e Vidal Center for Imaging Science, Johns Hopkins University, Baltimore MD 21218, USA Abstract In many problems in computer vision, data in multiple classes lie in multiple low-dimensional subspaces of a high- dimensional ambient space. However, most of the existing classification methods do not explicitly take this structure into account. In this paper, we consider the problem of clas- sification in the multi-subspace setting using sparse repre- sentation techniques. We exploit the fact that the dictio- nary of all the training data has a block structure where the training data in each class form few blocks of the dic- tionary. We cast the classification as a structured sparse recovery problem where our goal is to find a representation of a test example that uses the minimum number of blocks from the dictionary. We formulate this problem using two different classes of non-convex optimization programs. We propose convex relaxations for these two non-convex pro- grams and study conditions under which the relaxations are equivalent to the original problems. In addition, we show that the proposed optimization programs can be modified properly to also deal with corrupted data. To evaluate the proposed algorithms, we consider the problem of automatic face recognition. We show that casting the face recognition problem as a structured sparse recovery problem can im- prove the results of the state-of-the-art face recognition al- gorithms, especially when we have relatively small number of training data for each class. In particular, we show that the new class of convex programs can improve the state-of- the-art face recognition results by 10% with only 25% of the training data. In addition, we show that the algorithms are robust to occlusion, corruption, and disguise. 1. Introduction Classification is one of the most fundamental problems in machine learning and has numerous applications in dif- ferent areas including computer vision. Given training data from multiple classes, the task is to find the class to which a test example belongs. Recently, there has been an increasing interest in clas- sification problems where the data across multiple classes come from a collection of low-dimensional linear sub- B[k] B[1] B[n] Figure 1. In face recognition, the dictionary has a block structure where the training images of each subject form a few blocks of the dictionary. spaces. In fact, for many important problems in computer vision such as face recognition [13], motion segmentation [5], and activity recognition [14], the data lie in multiple low-dimensional subspaces of a high dimensional ambient space. However, most existing classification methods do not explicitly take into account the multi-subspace structure of the data. An important class of methods that deals with data on multiple subspaces relies on the notion of sparsity. Specif- ically, the sparse representation-based classification (SRC) method [13] looks for the sparsest representation of a test example in a dictionary composed of all training data across all classes. More formally, given a dictionary B and a test example y, it solves the following non-convex program P���0 : min kck0 s.t. y = Bc, where kck0 denotes the number of nonzero elements of c. Assuming that the underlying subspace for each class is low-dimensional, the sparsest representation of a test exam- ple ideally corresponds to the training data from the same class. When it comes to the problem of robust classification, the SRC method offers a great advantage over many classi- fication methods since it can effectively deal with corrupted data within the same sparse representation framework. Challenges. While sparse representation-based methods have been shown to be effective for classification, there still remain questions about classification in the multi-subspace setting using sparse representation which have not been suf- ficiently explored or have not been answered yet. C1��� The SRC method looks for the sparsest representation of a test example with the hope that such a representation selects few training data from the correct class. However, as shown in Figure 1, the dictionary of the training data has 1873