SigCLR: Sigmoid Contrastive Learning of Visual Representations
This work offers a potential replacement for the widely used SimCLR method in self-supervised learning, but it appears incremental as it modifies an existing loss function.
The paper tackles the problem of self-supervised visual representation learning by proposing SigCLR, which uses a logistic loss instead of cross-entropy, showing competitive performance on datasets like CIFAR-10, CIFAR-100, and Tiny-IN.
We propose SigCLR: Sigmoid Contrastive Learning of Visual Representations. SigCLR utilizes the logistic loss that only operates on pairs and does not require a global view as in the cross-entropy loss used in SimCLR. We show that logistic loss shows competitive performance on CIFAR-10, CIFAR-100, and Tiny-IN compared to other established SSL objectives. Our findings verify the importance of learnable bias as in the case of SigLUP, however, it requires a fixed temperature as in the SimCLR to excel. Overall, SigCLR is a promising replacement for the SimCLR which is ubiquitous and has shown tremendous success in various domains.