Adversarial Contrastive Learning via Asymmetric InfoNCE
This work addresses a specific problem in adversarial robustness for machine learning practitioners, offering an incremental improvement over existing methods.
The paper tackles the issue of adversarial perturbations causing identity confusion in contrastive learning by proposing an asymmetric InfoNCE objective that treats adversarial samples as inferior positives or hard negatives, resulting in consistent outperformance of existing adversarial contrastive learning methods across different finetuning schemes without extra computational cost.
Contrastive learning (CL) has recently been applied to adversarial learning tasks. Such practice considers adversarial samples as additional positive views of an instance, and by maximizing their agreements with each other, yields better adversarial robustness. However, this mechanism can be potentially flawed, since adversarial perturbations may cause instance-level identity confusion, which can impede CL performance by pulling together different instances with separate identities. To address this issue, we propose to treat adversarial samples unequally when contrasted, with an asymmetric InfoNCE objective ($A-InfoNCE$) that allows discriminating considerations of adversarial samples. Specifically, adversaries are viewed as inferior positives that induce weaker learning signals, or as hard negatives exhibiting higher contrast to other negative samples. In the asymmetric fashion, the adverse impacts of conflicting objectives between CL and adversarial learning can be effectively mitigated. Experiments show that our approach consistently outperforms existing Adversarial CL methods across different finetuning schemes without additional computational cost. The proposed A-InfoNCE is also a generic form that can be readily extended to other CL methods. Code is available at https://github.com/yqy2001/A-InfoNCE.