Accelerating Neural Self-Improvement via Bootstrapping
This work addresses improving efficiency in few-shot learning for AI applications, but it appears incremental as it builds on existing meta-learning methods.
The paper tackles the problem of accelerating few-shot learning in neural networks by introducing an auxiliary loss that uses bootstrapped meta-learning to match performance achievable with more examples using only the standard NK examples, showing promising results on Mini-ImageNet.
Few-shot learning with sequence-processing neural networks (NNs) has recently attracted a new wave of attention in the context of large language models. In the standard N-way K-shot learning setting, an NN is explicitly optimised to learn to classify unlabelled inputs by observing a sequence of NK labelled examples. This pressures the NN to learn a learning algorithm that achieves optimal performance, given the limited number of training examples. Here we study an auxiliary loss that encourages further acceleration of few-shot learning, by applying recently proposed bootstrapped meta-learning to NN few-shot learners: we optimise the K-shot learner to match its own performance achievable by observing more than NK examples, using only NK examples. Promising results are obtained on the standard Mini-ImageNet dataset. Our code is public.