Learning and Free Energies for Vector Approximate Message Passing
This work addresses the need for robust signal recovery in scenarios with unknown statistical parameters, though it is incremental as it builds on existing VAMP methods.
The authors tackled the problem of recovering signals from noisy linear measurements without prior knowledge of signal and noise statistics by combining Vector Approximate Message Passing (VAMP) with Expectation-Maximization to create EM-VAMP, which achieves performance nearly matching oracle-parameter VAMP in simulations with highly ill-conditioned matrices.
Vector approximate message passing (VAMP) is a computationally simple approach to the recovery of a signal $\mathbf{x}$ from noisy linear measurements $\mathbf{y}=\mathbf{Ax}+\mathbf{w}$. Like the AMP proposed by Donoho, Maleki, and Montanari in 2009, VAMP is characterized by a rigorous state evolution (SE) that holds under certain large random matrices and that matches the replica prediction of optimality. But while AMP's SE holds only for large i.i.d. sub-Gaussian $\mathbf{A}$, VAMP's SE holds under the much larger class: right-rotationally invariant $\mathbf{A}$. To run VAMP, however, one must specify the statistical parameters of the signal and noise. This work combines VAMP with Expectation-Maximization to yield an algorithm, EM-VAMP, that can jointly recover $\mathbf{x}$ while learning those statistical parameters. The fixed points of the proposed EM-VAMP algorithm are shown to be stationary points of a certain constrained free-energy, providing a variational interpretation of the algorithm. Numerical simulations show that EM-VAMP is robust to highly ill-conditioned $\mathbf{A}$ with performance nearly matching oracle-parameter VAMP.