0-1 phase transitions in sparse spiked matrix estimation
This work addresses the fundamental problem of sparse matrix estimation for statisticians and machine learning researchers, providing theoretical insights into phase transitions, but it is incremental as it builds on prior analyses in sparse linear regression.
The paper tackles the problem of estimating a sparse rank-one matrix (spike) corrupted by Gaussian noise, proving explicit variational formulas for asymptotic mutual information and showing sharp 0-1 phase transitions in the asymptotic minimum mean-square-error under specific sparsity and signal strength conditions.
We consider statistical models of estimation of a rank-one matrix (the spike) corrupted by an additive gaussian noise matrix in the sparse limit. In this limit the underlying hidden vector (that constructs the rank-one matrix) has a number of non-zero components that scales sub-linearly with the total dimension of the vector, and the signal strength tends to infinity at an appropriate speed. We prove explicit low-dimensional variational formulas for the asymptotic mutual information between the spike and the observed noisy matrix in suitable sparse limits. For Bernoulli and Bernoulli-Rademacher distributed vectors, and when the sparsity and signal strength satisfy an appropriate scaling relation, these formulas imply sharp 0-1 phase transitions for the asymptotic minimum mean-square-error. A similar phase transition was analyzed recently in the context of sparse high-dimensional linear regression (compressive sensing).