Contrastive Learning with Nasty Noise
It addresses the problem of adversarial robustness in self-supervised learning for researchers, but is incremental as it builds on existing theoretical frameworks.
This work analyzes the theoretical limits of contrastive learning under nasty noise, where an adversary modifies training samples, establishing lower and upper bounds on sample complexity using PAC learning and VC-dimension analysis.
Contrastive learning has emerged as a powerful paradigm for self-supervised representation learning. This work analyzes the theoretical limits of contrastive learning under nasty noise, where an adversary modifies or replaces training samples. Using PAC learning and VC-dimension analysis, lower and upper bounds on sample complexity in adversarial settings are established. Additionally, data-dependent sample complexity bounds based on the l2-distance function are derived.