OpenCon: Open-world Contrastive Learning
This addresses the challenge of learning from both labeled and unlabeled data in real-world deployments for machine learning practitioners, though it appears incremental as it builds on existing contrastive learning frameworks.
The paper tackles the problem of open-world semi-supervised learning, where models encounter unlabeled samples from known and novel classes, and demonstrates that OpenCon significantly outperforms the current best method by 11.9% and 7.4% on novel and overall classification accuracy on ImageNet.
Machine learning models deployed in the wild naturally encounter unlabeled samples from both known and novel classes. Challenges arise in learning from both the labeled and unlabeled data, in an open-world semi-supervised manner. In this paper, we introduce a new learning framework, open-world contrastive learning (OpenCon). OpenCon tackles the challenges of learning compact representations for both known and novel classes and facilitates novelty discovery along the way. We demonstrate the effectiveness of OpenCon on challenging benchmark datasets and establish competitive performance. On the ImageNet dataset, OpenCon significantly outperforms the current best method by 11.9% and 7.4% on novel and overall classification accuracy, respectively. Theoretically, OpenCon can be rigorously interpreted from an EM algorithm perspective--minimizing our contrastive loss partially maximizes the likelihood by clustering similar samples in the embedding space. The code is available at https://github.com/deeplearning-wisc/opencon.