Meta-Learning with Latent Embedding Optimization
This addresses a practical limitation in meta-learning for few-shot learning, offering a solution that improves adaptation efficiency in low-data regimes.
The paper tackles the challenge of gradient-based meta-learning in high-dimensional parameter spaces with very little data by introducing a latent generative representation of model parameters, enabling optimization in a lower-dimensional space. The resulting method, LEO, achieves state-of-the-art performance on miniImageNet and tieredImageNet few-shot classification tasks.
Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems. However, they have practical difficulties when operating on high-dimensional parameter spaces in extreme low-data regimes. We show that it is possible to bypass these limitations by learning a data-dependent latent generative representation of model parameters, and performing gradient-based meta-learning in this low-dimensional latent space. The resulting approach, latent embedding optimization (LEO), decouples the gradient-based adaptation procedure from the underlying high-dimensional space of model parameters. Our evaluation shows that LEO can achieve state-of-the-art performance on the competitive miniImageNet and tieredImageNet few-shot classification tasks. Further analysis indicates LEO is able to capture uncertainty in the data, and can perform adaptation more effectively by optimizing in latent space.