Rigorous Restricted Isometry Property of Low-Dimensional Subspaces
This provides a theoretical guarantee for dimensionality reduction in machine learning and computer vision, analogous to the Johnson-Lindenstrauss Lemma, but it is incremental as it extends existing RIP concepts to subspaces.
The paper tackles the problem of preserving distances between low-dimensional subspaces after random projection, proving that Gaussian random projections maintain the Frobenius norm distance between subspaces with high probability, specifically at least 1 - e^{-O(n)}.
Dimensionality reduction is in demand to reduce the complexity of solving large-scale problems with data lying in latent low-dimensional structures in machine learning and computer version. Motivated by such need, in this work we study the Restricted Isometry Property (RIP) of Gaussian random projections for low-dimensional subspaces in $\mathbb{R}^N$, and rigorously prove that the projection Frobenius norm distance between any two subspaces spanned by the projected data in $\mathbb{R}^n$ ($n<N$) remain almost the same as the distance between the original subspaces with probability no less than $1 - {\rm e}^{-\mathcal{O}(n)}$. Previously the well-known Johnson-Lindenstrauss (JL) Lemma and RIP for sparse vectors have been the foundation of sparse signal processing including Compressed Sensing. As an analogy to JL Lemma and RIP for sparse vectors, this work allows the use of random projections to reduce the ambient dimension with the theoretical guarantee that the distance between subspaces after compression is well preserved.