VIABLE: Fast Adaptation via Backpropagating Learned Loss
This work addresses the challenge of adapting quickly with limited data in meta-learning, offering a novel approach that could enhance few-shot learning applications, though it appears incremental as it builds on existing meta-gradient methods.
The paper tackles the problem of few-shot learning by proposing VIABLE, a meta-learning extension that learns a differentiable loss function to replace predefined inner-loop losses, resulting in reduced underfitting and significant improvements in performance and sample efficiency on a regression task and scalability on Mini-Imagenet.
In few-shot learning, typically, the loss function which is applied at test time is the one we are ultimately interested in minimising, such as the mean-squared-error loss for a regression problem. However, given that we have few samples at test time, we argue that the loss function that we are interested in minimising is not necessarily the loss function most suitable for computing gradients in a few-shot setting. We propose VIABLE, a generic meta-learning extension that builds on existing meta-gradient-based methods by learning a differentiable loss function, replacing the pre-defined inner-loop loss function in performing task-specific updates. We show that learning a loss function capable of leveraging relational information between samples reduces underfitting, and significantly improves performance and sample efficiency on a simple regression task. Furthermore, we show VIABLE is scalable by evaluating on the Mini-Imagenet dataset.