Rank/Norm Regularization with Closed-Form Solutions: Application to Subspace Clustering
This provides a theoretical advancement for researchers in machine learning and data analysis, offering new insights into subspace clustering, though it appears incremental as it builds on existing PCA frameworks.
The paper tackles the problem of subspace clustering by generalizing the Eckart-Young-Mirsky theorem to all unitarily invariant norms, leading to closed-form solutions for rank/norm regularized problems and improved experimental results.
When data is sampled from an unknown subspace, principal component analysis (PCA) provides an effective way to estimate the subspace and hence reduce the dimension of the data. At the heart of PCA is the Eckart-Young-Mirsky theorem, which characterizes the best rank k approximation of a matrix. In this paper, we prove a generalization of the Eckart-Young-Mirsky theorem under all unitarily invariant norms. Using this result, we obtain closed-form solutions for a set of rank/norm regularized problems, and derive closed-form solutions for a general class of subspace clustering problems (where data is modelled by unions of unknown subspaces). From these results we obtain new theoretical insights and promising experimental results.