Dataset Watermarking for Closed LLMs with Provable Detection
This work addresses the need to detect unauthorized use of proprietary datasets in closed LLMs, a problem for data owners and regulators.
The paper introduces the first dataset watermarking method for closed LLMs with provable detection, embedding watermarks via increased co-occurrence of random word pairs and detecting them with a statistical test. The method reliably detects watermarks (p < 0.01) even when the watermarked data is only 1% of fine-tuning tokens.
Large language models (LLMs) are pre-trained and post-trained on vast amounts of loosely curated data, raising the possibility that these models may have been trained on proprietary datasets or the same benchmarks used for evaluation. This motivates the need for dataset watermarking: designing datasets such that training on them leaves detectable signatures in the resulting model. Prior work has explored this problem for open models. We introduce the first dataset watermarking method for closed LLMs with provable detection. In particular, we embed a dataset-level watermark signal by increasing the co-occurrence frequency of randomly selected word pairs through rephrasing, and detect it using a statistical test on co-occurrence patterns in model-generated outputs. We evaluate our method with multiple base models and benchmark datasets and show that it reliably detects the watermark ($p <0.01$) in the fine-tuning stage. Notably, our method remains effective in a data mixture setting where the watermarked dataset constitutes only approximately $1\%$ of the total fine-tuning tokens. Furthermore, we show that our method preserves the utility and semantic integrity of the benchmark.