All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation
This addresses fundamental limits in sparse matrix estimation, relevant for high-dimensional statistics and machine learning, but is incremental as it builds on existing spiked matrix models.
The paper tackles the problem of estimating a sparse rank-one matrix corrupted by Gaussian noise, deriving exact asymptotic thresholds for statistical and computational limits. It proves all-orothing phase transitions where mean-square errors jump from maximum to zero at specific signal-to-noise ratios, with a diverging statistical-to-algorithmic gap indicating hardness for approximate message passing.
We determine statistical and computational limits for estimation of a rank-one matrix (the spike) corrupted by an additive gaussian noise matrix, in a sparse limit, where the underlying hidden vector (that constructs the rank-one matrix) has a number of non-zero components that scales sub-linearly with the total dimension of the vector, and the signal-to-noise ratio tends to infinity at an appropriate speed. We prove explicit low-dimensional variational formulas for the asymptotic mutual information between the spike and the observed noisy matrix and analyze the approximate message passing algorithm in the sparse regime. For Bernoulli and Bernoulli-Rademacher distributed vectors, and when the sparsity and signal strength satisfy an appropriate scaling relation, we find all-or-nothing phase transitions for the asymptotic minimum and algorithmic mean-square errors. These jump from their maximum possible value to zero, at well defined signal-to-noise thresholds whose asymptotic values we determine exactly. In the asymptotic regime the statistical-to-algorithmic gap diverges indicating that sparse recovery is hard for approximate message passing.