Concentration of the matrix-valued minimum mean-square error in optimal Bayesian inference
This work addresses a foundational issue in statistical inference for high-dimensional problems, such as spiked matrix models and neural networks, but it is incremental as it extends existing concentration techniques to a matrix-valued context.
The paper tackles the problem of Bayesian inference for signals with vector-valued entries by showing that the matrix-valued minimum mean-square error concentrates as problem size increases, using techniques from spin glass physics. This result is crucial for proving single-letter formulas for mutual information in optimal Bayesian inference settings.
We consider Bayesian inference of signals with vector-valued entries. Extending concentration techniques from the mathematical physics of spin glasses, we show that the matrix-valued minimum mean-square error concentrates when the size of the problem increases. Such results are often crucial for proving single-letter formulas for the mutual information when they exist. Our proof is valid in the optimal Bayesian inference setting, meaning that it relies on the assumption that the model and all its hyper-parameters are known. Examples of inference and learning problems covered by our results are spiked matrix and tensor models, the committee machine neural network with few hidden neurons in the teacher-student scenario, or multi-layers generalized linear models.