Data-Driven Priors in the Maximum Entropy on the Mean Method for Linear Inverse Problems
This work addresses linear inverse problems for researchers in computational mathematics and machine learning, but it is incremental as it builds upon existing maximum entropy methods.
The paper tackles the problem of linear inverse problems by establishing a theoretical framework for the maximum entropy on the mean method with data-driven priors, proving almost sure convergence for empirical means and providing convergence rates, illustrated with denoising on MNIST and Fashion-MNIST datasets.
We establish the theoretical framework for implementing the maximumn entropy on the mean (MEM) method for linear inverse problems in the setting of approximate (data-driven) priors. We prove a.s. convergence for empirical means and further develop general estimates for the difference between the MEM solutions with different priors $μ$ and $ν$ based upon the epigraphical distance between their respective log-moment generating functions. These estimates allow us to establish a rate of convergence in expectation for empirical means. We illustrate our results with denoising on MNIST and Fashion-MNIST data sets.