Regularization by Texts for Latent Diffusion Inverse Solvers
This addresses ambiguity challenges in inverse problems for researchers and practitioners using diffusion models, representing an incremental advance.
The paper tackles the ill-posed nature of inverse problems in diffusion models by introducing TReg, a latent diffusion inverse solver that uses textual descriptions for regularization, resulting in improved accuracy and efficiency in mitigating ambiguities.
The recent development of diffusion models has led to significant progress in solving inverse problems by leveraging these models as powerful generative priors. However, challenges persist due to the ill-posed nature of such problems, often arising from ambiguities in measurements or intrinsic system symmetries. To address this, here we introduce a novel latent diffusion inverse solver, regularization by text (TReg), inspired by the human ability to resolve visual ambiguities through perceptual biases. TReg integrates textual descriptions of preconceptions about the solution during reverse diffusion sampling, dynamically reinforcing these descriptions through null-text optimization, which we refer to as adaptive negation. Our comprehensive experimental results demonstrate that TReg effectively mitigates ambiguity in inverse problems, improving both accuracy and efficiency.