Watermarking Low-entropy Generation for Large Language Models: An Unbiased and Low-risk Method
This addresses the risk of LLM misuse by providing a low-risk watermarking solution, though it is incremental as it extends existing methods.
The paper tackles the problem of detecting LLM-generated content by proposing the STA-1 watermarking method, which preserves token distribution and reduces risk in low-entropy scenarios, achieving high efficiency and robustness in experiments.
Recent advancements in large language models (LLMs) have highlighted the risk of misusing them, raising the need for accurate detection of LLM-generated content. In response, a viable solution is to inject imperceptible identifiers into LLMs, known as watermarks. Our research extends the existing watermarking methods by proposing the novel Sampling One Then Accepting (STA-1) method. STA-1 is an unbiased watermark that preserves the original token distribution in expectation and has a lower risk of producing unsatisfactory outputs in low-entropy scenarios compared to existing unbiased watermarks. In watermark detection, STA-1 does not require prompts or a white-box LLM, provides statistical guarantees, demonstrates high efficiency in detection time, and remains robust against various watermarking attacks. Experimental results on low-entropy and high-entropy datasets demonstrate that STA-1 achieves the above properties simultaneously, making it a desirable solution for watermarking LLMs. Implementation codes for this study are available online.