HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks
This addresses the problem of inefficient adaptation in few-shot learning for AI researchers, offering an incremental improvement over MAML.
The paper tackles the limitation of Model-Agnostic Meta-Learning (MAML) in few-shot learning, where gradient-based updates may be insufficient or inefficient, by introducing HyperMAML, which uses a trainable hypernetwork to generate updates, resulting in consistent outperformance of MAML and competitive performance with state-of-the-art methods on standard benchmarks.
The aim of Few-Shot learning methods is to train models which can easily adapt to previously unseen tasks, based on small amounts of data. One of the most popular and elegant Few-Shot learning approaches is Model-Agnostic Meta-Learning (MAML). The main idea behind this method is to learn the general weights of the meta-model, which are further adapted to specific problems in a small number of gradient steps. However, the model's main limitation lies in the fact that the update procedure is realized by gradient-based optimisation. In consequence, MAML cannot always modify weights to the essential level in one or even a few gradient iterations. On the other hand, using many gradient steps results in a complex and time-consuming optimization procedure, which is hard to train in practice, and may lead to overfitting. In this paper, we propose HyperMAML, a novel generalization of MAML, where the training of the update procedure is also part of the model. Namely, in HyperMAML, instead of updating the weights with gradient descent, we use for this purpose a trainable Hypernetwork. Consequently, in this framework, the model can generate significant updates whose range is not limited to a fixed number of gradient steps. Experiments show that HyperMAML consistently outperforms MAML and performs comparably to other state-of-the-art techniques in a number of standard Few-Shot learning benchmarks.