Dataset Ownership Verification in Contrastive Pre-trained Models
This work addresses the problem of protecting dataset ownership for data owners, particularly in the context of unsupervised pre-trained models, which is an incremental yet important step in the field of dataset ownership verification.
The authors tackled the problem of dataset ownership verification in contrastive pre-trained models, achieving a significant result with a p-value below 0.05, outperforming previous methodologies. The approach was validated across multiple models including SimCLR, BYOL, SimSiam, MOCO v3, and DINO.
High-quality open-source datasets, which necessitate substantial efforts for curation, has become the primary catalyst for the swift progress of deep learning. Concurrently, protecting these datasets is paramount for the well-being of the data owner. Dataset ownership verification emerges as a crucial method in this domain, but existing approaches are often limited to supervised models and cannot be directly extended to increasingly popular unsupervised pre-trained models. In this work, we propose the first dataset ownership verification method tailored specifically for self-supervised pre-trained models by contrastive learning. Its primary objective is to ascertain whether a suspicious black-box backbone has been pre-trained on a specific unlabeled dataset, aiding dataset owners in upholding their rights. The proposed approach is motivated by our empirical insights that when models are trained with the target dataset, the unary and binary instance relationships within the embedding space exhibit significant variations compared to models trained without the target dataset. We validate the efficacy of this approach across multiple contrastive pre-trained models including SimCLR, BYOL, SimSiam, MOCO v3, and DINO. The results demonstrate that our method rejects the null hypothesis with a $p$-value markedly below $0.05$, surpassing all previous methodologies. Our code is available at https://github.com/xieyc99/DOV4CL.