Multi-Level Contrastive Learning for Few-Shot Problems
This work addresses few-shot learning problems for computer vision applications, representing an incremental improvement over existing contrastive learning methods.
The paper tackled the problem of few-shot learning by proposing a multi-level contrastive learning approach that applies contrastive losses at different encoder layers to learn multiple representations, achieving new state-of-the-art results on mini-ImageNet and tiered-ImageNet datasets.
Contrastive learning is a discriminative approach that aims at grouping similar samples closer and diverse samples far from each other. It it an efficient technique to train an encoder generating distinguishable and informative representations, and it may even increase the encoder's transferability. Most current applications of contrastive learning benefit only a single representation from the last layer of an encoder.In this paper, we propose a multi-level contrasitive learning approach which applies contrastive losses at different layers of an encoder to learn multiple representations from the encoder. Afterward, an ensemble can be constructed to take advantage of the multiple representations for the downstream tasks. We evaluated the proposed method on few-shot learning problems and conducted experiments using the mini-ImageNet and the tiered-ImageNet datasets. Our model achieved the new state-of-the-art results for both datasets, comparing to previous regular, ensemble, and contrastive learing (single-level) based approaches.