On the role of memorization in learned priors for geophysical inverse problems
This addresses a critical issue for geoscientists using data-driven regularization in seismic inversion, where scarce training data can lead to poor generalization.
The paper tackles the problem of deep generative models memorizing training data instead of learning the underlying geological distribution in seismic inversion, showing that this leads to a posterior distribution that is a likelihood-weighted lookup among training examples. They validate this on a stylized inverse problem and demonstrate its consequences in full waveform inversion.
Learned priors based on deep generative models offer data-driven regularization for seismic inversion, but training them requires a dataset of representative subsurface models -- a resource that is inherently scarce in geoscience applications. Since the training objective of most generative models can be cast as maximum likelihood on a finite dataset, any such model risks converging to the empirical distribution -- effectively memorizing the training examples rather than learning the underlying geological distribution. We show that the posterior under such a memorized prior reduces to a reweighted empirical distribution -- i.e., a likelihood-weighted lookup among the stored training examples. For diffusion models specifically, memorization yields a Gaussian mixture prior in closed form, and linearizing the forward operator around each training example gives a Gaussian mixture posterior whose components have widths and shifts governed by the local Jacobian. We validate these predictions on a stylized inverse problem and demonstrate the consequences of memorization through diffusion posterior sampling for full waveform inversion.