ATHENA: Adaptive Test-Time Steering for Improving Count Fidelity in Diffusion Models
This addresses a specific limitation in diffusion models for users needing precise numerical control in image generation, representing an incremental improvement.
The paper tackled the problem of text-to-image diffusion models failing to accurately generate specified object counts in prompts, and introduced ATHENA, a test-time adaptive steering framework that improved count fidelity without retraining, with experiments showing consistent gains especially at higher counts.
Text-to-image diffusion models achieve high visual fidelity but surprisingly exhibit systematic failures in numerical control when prompts specify explicit object counts. To address this limitation, we introduce ATHENA, a model-agnostic, test-time adaptive steering framework that improves object count fidelity without modifying model architectures or requiring retraining. ATHENA leverages intermediate representations during sampling to estimate object counts and applies count-aware noise corrections early in the denoising process, steering the generation trajectory before structural errors become difficult to revise. We present three progressively more advanced variants of ATHENA that trade additional computation for improved numerical accuracy, ranging from static prompt-based steering to dynamically adjusted count-aware control. Experiments on established benchmarks and a new visually and semantically complex dataset show that ATHENA consistently improves count fidelity, particularly at higher target counts, while maintaining favorable accuracy-runtime trade-offs across multiple diffusion backbones.