CATMark: A Context-Aware Thresholding Framework for Robust Cross-Task Watermarking in Large Language Models
This work addresses the challenge of robust and adaptable watermarking for LLMs, which is crucial for detecting machine-generated content in applications like content moderation and authentication, though it appears incremental as it builds on existing entropy-based methods.
The paper tackles the problem of text quality degradation in watermarking algorithms for Large Language Models (LLMs) during low-entropy scenarios, proposing a context-aware thresholding framework that dynamically adjusts watermarking intensity based on semantic context, resulting in improved text quality in cross-task scenarios without sacrificing detection accuracy.
Watermarking algorithms for Large Language Models (LLMs) effectively identify machine-generated content by embedding and detecting hidden statistical features in text. However, such embedding leads to a decline in text quality, especially in low-entropy scenarios where performance needs improvement. Existing methods that rely on entropy thresholds often require significant computational resources for tuning and demonstrate poor adaptability to unknown or cross-task generation scenarios. We propose \textbf{C}ontext-\textbf{A}ware \textbf{T}hreshold watermarking ($\myalgo$), a novel framework that dynamically adjusts watermarking intensity based on real-time semantic context. $\myalgo$ partitions text generation into semantic states using logits clustering, establishing context-aware entropy thresholds that preserve fidelity in structured content while embedding robust watermarks. Crucially, it requires no pre-defined thresholds or task-specific tuning. Experiments show $\myalgo$ improves text quality in cross-tasks without sacrificing detection accuracy.