Watermarking for Proprietary Dataset Protection
It offers an alternative approach for dataset membership inference in generative models, though it is incremental and relies on specific assumptions.
The paper proposes using output watermarking to improve membership inference for generative models, showing that watermark-based detection achieves comparable performance to loss-based methods when subset exposure is high.
A growing body of literature suggests that training data membership inference problems are fundamentally hard tasks in modern language modeling settings. We argue that output watermarking techniques are the right gadget to make training membership tests for generative models more tractable, based on prior results showing that language models exhibit residual watermark "radioactivity" under partially watermarked training datasets. We pit a watermark-based dataset inference approach head-to-head against traditional loss-based membership inference methods and show that watermarking can achieve comparable membership detection performance when subset exposure is high enough, under an alternate set of assumptions.