Deep Prior
This work addresses the challenge of improving generalization and extrapolation in meta-learning for tasks with limited data, though it appears incremental as it builds on existing variational Bayes and deep learning methods.
The paper tackles the problem of learning a prior distribution over neural network parameters using deep learning tools, resulting in a variational Bayes algorithm that generalizes well to new tasks with few training examples and correctly extrapolates far from training data on a meta-dataset of periodic signals.
The recent literature on deep learning offers new tools to learn a rich probability distribution over high dimensional data such as images or sounds. In this work we investigate the possibility of learning the prior distribution over neural network parameters using such tools. Our resulting variational Bayes algorithm generalizes well to new tasks, even when very few training examples are provided. Furthermore, this learned prior allows the model to extrapolate correctly far from a given task's training data on a meta-dataset of periodic signals.