SyncTweedies: A General Generative Framework Based on Synchronized Diffusions
This work addresses the need for high-quality generative models in visual content creation, though it appears incremental as it builds on existing diffusion methods.
The authors tackled the problem of generating diverse visual content by synchronizing multiple diffusion processes, and found that averaging Tweedie's formula outputs across instance spaces yields superior quality with broad applicability.
We introduce a general framework for generating diverse visual content, including ambiguous images, panorama images, mesh textures, and Gaussian splat textures, by synchronizing multiple diffusion processes. We present exhaustive investigation into all possible scenarios for synchronizing multiple diffusion processes through a canonical space and analyze their characteristics across applications. In doing so, we reveal a previously unexplored case: averaging the outputs of Tweedie's formula while conducting denoising in multiple instance spaces. This case also provides the best quality with the widest applicability to downstream tasks. We name this case SyncTweedies. In our experiments generating visual content aforementioned, we demonstrate the superior quality of generation by SyncTweedies compared to other synchronization methods, optimization-based and iterative-update-based methods.