Semi-supervised Learning with Contrastive Predicative Coding
This work addresses scalability issues in semi-supervised learning for real-world applications, though it appears incremental as it adapts an existing technique to a known bottleneck.
The paper tackled the problem of inflexibility and inefficiency in semi-supervised learning by applying contrastive predictive coding to improve discriminative power with limited labels, resulting in scalable models like cpc-SSL and ccpc-SSL that perform well on large datasets such as ImageNet.
Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, many of them have thus far been either inflexible, inefficient or non-scalable. This paper explores recently developed contrastive predictive coding technique to improve discriminative power of deep learning models when a large portion of labels are absent. Two models, cpc-SSL and a class conditional variant~(ccpc-SSL) are presented. They effectively exploit the unlabeled data by extracting shared information between different parts of the (high-dimensional) data. The proposed approaches are inductive, and scale well to very large datasets like ImageNet, making them good candidates in real-world large scale applications.