Watermarking Discrete Diffusion Language Models
This addresses the need to track AI-generated content from emerging discrete diffusion models, which are popular for high inference throughput, but the work is incremental as it adapts existing watermarking ideas to a new model type.
The paper tackles the problem of watermarking discrete diffusion language models, which lack prior methods, by introducing a scheme that uses the Gumbel-max trick and sequence indexing for detection, achieving reliable detectability and proving distortion-free performance with exponentially decaying false detection probability.
Watermarking has emerged as a promising technique to track AI-generated content and differentiate it from authentic human creations. While prior work extensively studies watermarking for autoregressive large language models (LLMs) and image diffusion models, none address discrete diffusion language models, which are becoming popular due to their high inference throughput. In this paper, we introduce the first watermarking method for discrete diffusion models by applying the distribution-preserving Gumbel-max trick at every diffusion step and seeding the randomness with the sequence index to enable reliable detection. We experimentally demonstrate that our scheme is reliably detectable on state-of-the-art diffusion language models and analytically prove that it is distortion-free with an exponentially decaying probability of false detection in the token sequence length.