Multiscale Sparse Microcanonical Models
This work addresses the computational inefficiency of sampling maximum entropy processes in statistical modeling, offering a method for approximating complex processes in fields like image and audio processing, though it appears incremental as it builds on existing microcanonical and gradient descent frameworks.
The paper tackles the challenge of approximating non-Gaussian stationary processes with long-range correlations by developing multiscale sparse microcanonical models, using gradient descent to enforce energy constraints for efficient sampling, and demonstrates applications in texture synthesis for images and audio.
We study approximations of non-Gaussian stationary processes having long range correlations with microcanonical models. These models are conditioned by the empirical value of an energy vector, evaluated on a single realization. Asymptotic properties of maximum entropy microcanonical and macrocanonical processes and their convergence to Gibbs measures are reviewed. We show that the Jacobian of the energy vector controls the entropy rate of microcanonical processes. Sampling maximum entropy processes through MCMC algorithms require too many operations when the number of constraints is large. We define microcanonical gradient descent processes by transporting a maximum entropy measure with a gradient descent algorithm which enforces the energy conditions. Convergence and symmetries are analyzed. Approximations of non-Gaussian processes with long range interactions are defined with multiscale energy vectors computed with wavelet and scattering transforms. Sparsity properties are captured with $\bf l^1$ norms. Approximations of Gaussian, Ising and point processes are studied, as well as image and audio texture synthesis.