Few-Shot Learning with Siamese Networks and Label Tuning
This work addresses the challenge of few-shot text classification for researchers and practitioners, offering a more efficient alternative to existing methods, though it is incremental in nature.
The paper tackles the problem of building text classifiers with minimal training data by proposing Siamese Networks with label tuning, achieving competitive performance while reducing inference cost to constant in the number of labels.
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification. In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. In this work, we show that with proper pre-training, Siamese Networks that embed texts and labels offer a competitive alternative. These models allow for a large reduction in inference cost: constant in the number of labels rather than linear. Furthermore, we introduce label tuning, a simple and computationally efficient approach that allows to adapt the models in a few-shot setup by only changing the label embeddings. While giving lower performance than model fine-tuning, this approach has the architectural advantage that a single encoder can be shared by many different tasks.