Watermarking Pre-trained Encoders in Contrastive Learning
This addresses the need for intellectual property protection for pre-trained encoders in contrastive learning, which is an incremental advancement in watermarking techniques.
The paper tackles the problem of protecting pre-trained image encoders in contrastive learning as intellectual property by proposing the first watermarking methodology, which embeds a backdoor as a watermark that persists in downstream models with high effectiveness and robustness across various algorithms, datasets, and tasks.
Contrastive learning has become a popular technique to pre-train image encoders, which could be used to build various downstream classification models in an efficient way. This process requires a large amount of data and computation resources. Hence, the pre-trained encoders are an important intellectual property that needs to be carefully protected. It is challenging to migrate existing watermarking techniques from the classification tasks to the contrastive learning scenario, as the owner of the encoder lacks the knowledge of the downstream tasks which will be developed from the encoder in the future. We propose the \textit{first} watermarking methodology for the pre-trained encoders. We introduce a task-agnostic loss function to effectively embed into the encoder a backdoor as the watermark. This backdoor can still exist in any downstream models transferred from the encoder. Extensive evaluations over different contrastive learning algorithms, datasets, and downstream tasks indicate our watermarks exhibit high effectiveness and robustness against different adversarial operations.