Nuclear Norm Regularized Structural Orthogonal Procrustes Regression for Face Hallucination with Pose

Abstract

In real applications, the observed low-resolution (LR) face images usually have pose variations. Conventional learning based methods ignore these variations, thus the learned representations are not beneficial for the following reconstruction. In this paper, we propose a nuclear norm regularized structural orthogonal Procrustes regression (N2SOPR) method to learn pose-robust feature representations for efficient face hallucination. The orthogonal Procrustes regression (OPR) seeks an optimal transformation between two images to correct the pose from one to the other. Additionally, our N2SOPR uses the nuclear norm constraint on the error term to keep image’s structural information. A low-rank constraint on the representation coefficients is imposed to adaptively select the training samples that belong to the same subspace as the inputs. Moreover, a locality constraint is also enforced to preserve the locality and the sparsity simultaneously. Experimental results on standard face hallucination databases indicate that our proposed method can produce more reasonable near frontal face images for recognition purpose.