Learning a Gaussian Mixture for Sparsity Regularization in Inverse Problems
This work addresses the need for more accurate regularization in inverse problems for fields like signal processing, though it appears incremental as it builds on existing sparsity prior concepts with a novel probabilistic formulation.
The authors tackled the problem of improving reconstruction accuracy in linear inverse problems by proposing a Gaussian mixture sparsity prior and a corresponding neural network Bayes estimator. Their method achieved consistently lower mean square error compared to existing sparsity-promoting techniques like LASSO and iterative hard thresholding across all 1D datasets tested.
In inverse problems, it is widely recognized that the incorporation of a sparsity prior yields a regularization effect on the solution. This approach is grounded on the a priori assumption that the unknown can be appropriately represented in a basis with a limited number of significant components, while most coefficients are close to zero. This occurrence is frequently observed in real-world scenarios, such as with piecewise smooth signals. In this study, we propose a probabilistic sparsity prior formulated as a mixture of degenerate Gaussians, capable of modeling sparsity with respect to a generic basis. Under this premise, we design a neural network that can be interpreted as the Bayes estimator for linear inverse problems. Additionally, we put forth both a supervised and an unsupervised training strategy to estimate the parameters of this network. To evaluate the effectiveness of our approach, we conduct a numerical comparison with commonly employed sparsity-promoting regularization techniques, namely LASSO, group LASSO, iterative hard thresholding, and sparse coding/dictionary learning. Notably, our reconstructions consistently exhibit lower mean square error values across all $1$D datasets utilized for the comparisons, even in cases where the datasets significantly deviate from a Gaussian mixture model.