Meta-learning autoencoders for few-shot prediction
This addresses the challenge of few-shot learning for AI systems, enabling better generalization to new tasks with minimal data, though it appears incremental as it builds on existing meta-learning approaches.
The paper tackles the problem of machine learning models requiring many training examples and failing to generalize to unseen tasks by introducing a meta-learning autoencoder framework that learns succinct model codes from few examples and constructs task-specific parameters, resulting in significantly lower loss than fine-tuned baselines and competitive performance with state-of-the-art meta-learning algorithms.
Compared to humans, machine learning models generally require significantly more training examples and fail to extrapolate from experience to solve previously unseen challenges. To help close this performance gap, we augment single-task neural networks with a meta-recognition model which learns a succinct model code via its autoencoder structure, using just a few informative examples. The model code is then employed by a meta-generative model to construct parameters for the task-specific model. We demonstrate that for previously unseen tasks, without additional training, this Meta-Learning Autoencoder (MeLA) framework can build models that closely match the true underlying models, with loss significantly lower than given by fine-tuned baseline networks, and performance that compares favorably with state-of-the-art meta-learning algorithms. MeLA also adds the ability to identify influential training examples and predict which additional data will be most valuable to acquire to improve model prediction.