Tiny Inference-Time Scaling with Latent Verifiers
This work addresses efficiency bottlenecks in inference-time scaling for generative models, offering a domain-specific solution that is incremental but provides concrete computational savings.
The paper tackles the high computational cost of using Multimodal Large Language Models (MLLMs) as verifiers in inference-time scaling for generative models by proposing Verifier on Hidden States (VHS), which operates directly on intermediate hidden representations, reducing joint generation-and-verification time by 63.3%, compute FLOPs by 51%, and VRAM usage by 14.5% while improving performance by +2.7% on GenEval.
Inference-time scaling has emerged as an effective way to improve generative models at test time by using a verifier to score and select candidate outputs. A common choice is to employ Multimodal Large Language Models (MLLMs) as verifiers, which can improve performance but introduce substantial inference-time cost. Indeed, diffusion pipelines operate in an autoencoder latent space to reduce computation, yet MLLM verifiers still require decoding candidates to pixel space and re-encoding them into the visual embedding space, leading to redundant and costly operations. In this work, we propose Verifier on Hidden States (VHS), a verifier that operates directly on intermediate hidden representations of Diffusion Transformer (DiT) single-step generators. VHS analyzes generator features without decoding to pixel space, thereby reducing the per-candidate verification cost while improving or matching the performance of MLLM-based competitors. We show that, under tiny inference budgets with only a small number of candidates per prompt, VHS enables more efficient inference-time scaling reducing joint generation-and-verification time by 63.3%, compute FLOPs by 51% and VRAM usage by 14.5% with respect to a standard MLLM verifier, achieving a +2.7% improvement on GenEval at the same inference-time budget.