BaseTransformers: Attention over base data-points for One Shot Learning
This addresses performance degradation in few-shot classification for AI systems when test distributions differ from training, though it is incremental.
The paper tackles the problem of poor feature representations for novel classes in few-shot learning by attending to relevant base dataset features, achieving state-of-the-art results in inductive one-shot settings on three benchmarks.
Few shot classification aims to learn to recognize novel categories using only limited samples per category. Most current few shot methods use a base dataset rich in labeled examples to train an encoder that is used for obtaining representations of support instances for novel classes. Since the test instances are from a distribution different to the base distribution, their feature representations are of poor quality, degrading performance. In this paper we propose to make use of the well-trained feature representations of the base dataset that are closest to each support instance to improve its representation during meta-test time. To this end, we propose BaseTransformers, that attends to the most relevant regions of the base dataset feature space and improves support instance representations. Experiments on three benchmark data sets show that our method works well for several backbones and achieves state-of-the-art results in the inductive one shot setting. Code is available at github.com/mayug/BaseTransformers