Hypernetwork approach to Bayesian MAML
This addresses uncertainty estimation in Few-Shot learning, which is incremental as it builds on MAML with Bayesian enhancements.
The paper tackles the issues of over-fitting and poor uncertainty quantification in Model-Agnostic Meta-Learning (MAML) for Few-Shot learning by proposing BayesianHMAML, a Bayesian framework that uses Hypernetworks for weight updates, enabling the use of complex posteriors like Continuous Normalizing Flows.
The main goal of Few-Shot learning algorithms is to enable learning from small amounts of data. One of the most popular and elegant Few-Shot learning approaches is Model-Agnostic Meta-Learning (MAML). The main idea behind this method is to learn the shared universal weights of a meta-model, which are then adapted for specific tasks. However, the method suffers from over-fitting and poorly quantifies uncertainty due to limited data size. Bayesian approaches could, in principle, alleviate these shortcomings by learning weight distributions in place of point-wise weights. Unfortunately, previous modifications of MAML are limited due to the simplicity of Gaussian posteriors, MAML-like gradient-based weight updates, or by the same structure enforced for universal and adapted weights. In this paper, we propose a novel framework for Bayesian MAML called BayesianHMAML, which employs Hypernetworks for weight updates. It learns the universal weights point-wise, but a probabilistic structure is added when adapted for specific tasks. In such a framework, we can use simple Gaussian distributions or more complicated posteriors induced by Continuous Normalizing Flows.