Global Sketch-Based Watermarking for Diffusion Language Models
This work addresses the need for effective watermarking in diffusion language models, which are not well-served by existing autoregressive watermarking techniques.
The authors propose a global sketch-based watermarking method for masked diffusion language models, enabling order-agnostic detection without local token bias. The method achieves strong robustness and soundness with minimal distortion.
Watermarking methods for language models have been studied extensively in the autoregressive setting, where tokens are generated sequentially. These works largely focus on local-context schemes that perturb the next token's distribution as a function of its preceding tokens. In diffusion language models, distributions over many unresolved positions are jointly sampled, allowing additive statistics of the entire sequence to be tractable during generation. We propose a watermark for masked diffusion language models that controls a global, vector-valued sketch representation of the text. Compared to context-dependent watermarking, the sketch formulation decouples detection from the local contexts seen during generation, resulting in an order-agnostic statistic and a watermarking rule which does not manifest as a simple token bias. We analyze the distortion, soundness, and robustness properties of the method.