Non-Convex Rank Minimization via an Empirical Bayesian Approach
This addresses the challenge of correctly estimating low-rank matrices in applications like robust PCA, where standard convex relaxations often fail, offering a potentially more reliable solution for machine learning and data analysis tasks.
The paper tackles the problem of non-convex rank minimization in matrix solutions, proposing an empirical Bayesian approach that retains the same globally minimizing point estimate as the rank function under many constraints, with empirical evidence showing superiority over related MAP-based methods like convex principal component pursuit in robust principal component analysis.
In many applications that require matrix solutions of minimal rank, the underlying cost function is non-convex leading to an intractable, NP-hard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many practical scenarios there is no longer any guarantee that we can correctly estimate generative low-rank matrices of interest, theoretical special cases notwithstanding. Consequently, this paper proposes an alternative empirical Bayesian procedure build upon a variational approximation that, unlike the nuclear norm, retains the same globally minimizing point estimate as the rank function under many useful constraints. However, locally minimizing solutions are largely smoothed away via marginalization, allowing the algorithm to succeed when standard convex relaxations completely fail. While the proposed methodology is generally applicable to a wide range of low-rank applications, we focus our attention on the robust principal component analysis problem (RPCA), which involves estimating an unknown low-rank matrix with unknown sparse corruptions. Theoretical and empirical evidence are presented to show that our method is potentially superior to related MAP-based approaches, for which the convex principle component pursuit (PCP) algorithm (Candes et al., 2011) can be viewed as a special case.