The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generation
This work addresses the challenge of fine-grained control in diffusion-based image generation for users, offering a model-agnostic solution that is incremental by building on existing noise optimization methods.
The paper tackles the problem of improving quality and controllability in text-to-image generation by introducing NoiseQuery, a method that uses aligned Gaussian noise as implicit guidance alongside text prompts, resulting in significant performance boosts for both high-level semantics and low-level visual attributes with minimal computational overhead.
In this work, we introduce NoiseQuery as a novel method for enhanced noise initialization in versatile goal-driven text-to-image (T2I) generation. Specifically, we propose to leverage an aligned Gaussian noise as implicit guidance to complement explicit user-defined inputs, such as text prompts, for better generation quality and controllability. Unlike existing noise optimization methods designed for specific models, our approach is grounded in a fundamental examination of the generic finite-step noise scheduler design in diffusion formulation, allowing better generalization across different diffusion-based architectures in a tuning-free manner. This model-agnostic nature allows us to construct a reusable noise library compatible with multiple T2I models and enhancement techniques, serving as a foundational layer for more effective generation. Extensive experiments demonstrate that NoiseQuery enables fine-grained control and yields significant performance boosts not only over high-level semantics but also over low-level visual attributes, which are typically difficult to specify through text alone, with seamless integration into current workflows with minimal computational overhead.