HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Models
This addresses data ownership protection for users of generative diffusion models, offering a novel method for watermarking with high accuracy.
The paper tackles the problem of unauthorized use of training data in diffusion models by proposing HMARK, a multi-bit watermarking scheme that encodes ownership information in the semantic-latent space, achieving 98.57% detection accuracy and 95.07% bit-level recovery accuracy.
Modern generative diffusion models rely on vast training datasets, often including images with uncertain ownership or usage rights. Radioactive watermarks -- marks that transfer to a model's outputs -- can help detect when such unauthorized data has been used for training. Moreover, aside from being radioactive, an effective watermark for protecting images from unauthorized training also needs to meet other existing requirements, such as imperceptibility, robustness, and multi-bit capacity. To overcome these challenges, we propose HMARK, a novel multi-bit watermarking scheme, which encodes ownership information as secret bits in the semantic-latent space (h-space) for image diffusion models. By leveraging the interpretability and semantic significance of h-space, ensuring that watermark signals correspond to meaningful semantic attributes, the watermarks embedded by HMARK exhibit radioactivity, robustness to distortions, and minimal impact on perceptual quality. Experimental results demonstrate that HMARK achieves 98.57% watermark detection accuracy, 95.07% bit-level recovery accuracy, 100% recall rate, and 1.0 AUC on images produced by the downstream adversarial model finetuned with LoRA on watermarked data across various types of distortions.