Perturbation of Linear Forms of Singular Vectors Under Gaussian Noise

Abstract

Let A∈ ℝm×n be a matrix of rank r with singular value decomposition (SVD) A = ∑k = 1rσk(uk ⊗ vk), where {σk, k = 1, …, r} are singular values of A (arranged in a non-increasing order) and uk∈ ℝm, vk∈ ℝn, k= 1, …, r are the corresponding left and right orthonormal singular vectors. Let Ã= A+ X be a noisy observation of A, where X∈ ℝm×n is a random matrix with i.i.d. Gaussian entries, Xij∼ N(0, τ2), and consider its SVD Ã=∑k=1m∧nσ̃k(ũk⊗ṽk) with singular values σ̃1≥ … ≥ σ̃m∧n and singular vectors ũk, ṽk, k= 1, …, m∧ n. The goal of this paper is to develop sharp concentration bounds for linear forms ⟨ ũk, x⟩, x∈ ℝm and ⟨ ṽk, y⟩, y∈ ℝn of the perturbed (empirical) singular vectors in the case when the singular values of A are distinct and, more generally, concentration bounds for bilinear forms of projection operators associated with SVD. In particular, the results imply upper bounds of the order O(log(m+n)m∨n) (holding with a high probability) on max1≤i≤m| |andmax1≤j≤n| |, where bk are properly chosen constants characterizing the bias of empirical singular vectors ũk, ṽk and {eim, i = 1, …, m}, {ejn, j = 1, …, n} are the canonical bases of ℝm, ℝn, respectively.