Adaptive Testing for Segmenting Watermarked Texts From Language Models
This addresses the challenge of mitigating misinformation and misuse in education by improving detection of LLM-generated text, though it is incremental as it builds on prior watermark detection methods.
The paper tackles the problem of segmenting text into watermarked and non-watermarked substrings to distinguish LLM-generated content from human-written text, and the result is an effective and robust methodology that removes the need for precise prompt estimation.
The rapid adoption of large language models (LLMs), such as GPT-4 and Claude 3.5, underscores the need to distinguish LLM-generated text from human-written content to mitigate the spread of misinformation and misuse in education. One promising approach to address this issue is the watermark technique, which embeds subtle statistical signals into LLM-generated text to enable reliable identification. In this paper, we first generalize the likelihood-based LLM detection method of a previous study by introducing a flexible weighted formulation, and further adapt this approach to the inverse transform sampling method. Moving beyond watermark detection, we extend this adaptive detection strategy to tackle the more challenging problem of segmenting a given text into watermarked and non-watermarked substrings. In contrast to the approach in a previous study, which relies on accurate estimation of next-token probabilities that are highly sensitive to prompt estimation, our proposed framework removes the need for precise prompt estimation. Extensive numerical experiments demonstrate that the proposed methodology is both effective and robust in accurately segmenting texts containing a mixture of watermarked and non-watermarked content.