Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models
For users of diffusion LLMs needing format-constrained generation (e.g., JSON, reasoning templates), DIA offers a training-free solution to improve structural correctness and semantic coherence.
Dynamic Infilling Anchors (DIA) is a training-free method for diffusion LLMs that dynamically adjusts generation length to enforce format constraints, improving format compliance and answer accuracy with significant zero-shot gains on GSM8K and MATH.
Diffusion large language models (dLLMs) offer bidirectional attention and parallel generation, enabling them to exploit global context and naturally support format-constrained tasks like parseable JSON or reasoning templates. While straightforward fixed anchors can enforce such constraints, they often impose rigid spans, leading to truncated reasoning or redundant content. To overcome this, we propose Dynamic Infilling Anchors (DIA), a training-free method that dynamically estimates end-anchor positions to adjust generation length before iterative infilling. This flexible mechanism ensures structural correctness and semantic coherence, avoiding the inefficiencies of fixed-span methods. Experiments on reasoning benchmarks demonstrate that DIA substantially improves format compliance and answer accuracy, achieving significant zero-shot gains on GSM8K and MATH. These results establish DIA as a robust pathway toward reliable, structure-aware generation.