Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks
This work addresses the lack of understanding in meta-learning mechanics for few-shot tasks, offering insights and a practical improvement for researchers and practitioners in machine learning.
The paper investigates why meta-learned feature extractors excel in few-shot classification, proposing and verifying hypotheses about their mechanics and differences from classically trained models. It introduces a regularizer that boosts standard training performance, often outperforming meta-learning while being significantly faster.
Meta-learning algorithms produce feature extractors which achieve state-of-the-art performance on few-shot classification. While the literature is rich with meta-learning methods, little is known about why the resulting feature extractors perform so well. We develop a better understanding of the underlying mechanics of meta-learning and the difference between models trained using meta-learning and models which are trained classically. In doing so, we introduce and verify several hypotheses for why meta-learned models perform better. Furthermore, we develop a regularizer which boosts the performance of standard training routines for few-shot classification. In many cases, our routine outperforms meta-learning while simultaneously running an order of magnitude faster.