AnCoder: Anchored Code Generation via Discrete Diffusion Models
This addresses the issue of unreliable code generation for developers and AI assistants, though it is incremental as it builds on existing diffusion models with structural enhancements.
The paper tackled the problem of diffusion language models producing broken programs in code generation by introducing AnchorTree, a framework that uses abstract syntax trees to anchor the diffusion process with hierarchical priors, resulting in AnCoder models that achieve high-quality code generation with parameter efficiency.
Diffusion language models offer a compelling alternative to autoregressive code generation, enabling global planning and iterative refinement of complex program logic. However, existing approaches fail to respect the rigid structure of programming languages and, as a result, often produce broken programs that fail to execute. To address this, we introduce AnchorTree, a framework that explicitly anchors the diffusion process using structured, hierarchical priors native to code. Specifically, AnchorTree uses the abstract syntax tree to prioritize resolving syntactically and semantically salient tokens, such as keywords (e.g., if, while) and identifiers (e.g., variable names), thereby establishing a structural scaffold that guides the remaining generation. We validate this framework via AnCoder, a family of models showing that structurally anchored diffusion offers a parameter-efficient path to high-quality code generation.