Generalized Reinforcement Meta Learning for Few-Shot Optimization
This work addresses the problem of few-shot optimization for machine learning practitioners, offering a flexible framework that is incremental in combining RL with meta-learning.
The paper tackles few-shot learning by proposing a reinforcement learning-based meta-learning framework that learns optimization algorithms to improve learner performance, achieving accuracy and NDCG gains of up to 21% on datasets like MQ2007 and AwA2, with comparable results to Prototypical Networks on Mini-ImageNet.
We present a generic and flexible Reinforcement Learning (RL) based meta-learning framework for the problem of few-shot learning. During training, it learns the best optimization algorithm to produce a learner (ranker/classifier, etc) by exploiting stable patterns in loss surfaces. Our method implicitly estimates the gradients of a scaled loss function while retaining the general properties intact for parameter updates. Besides providing improved performance on few-shot tasks, our framework could be easily extended to do network architecture search. We further propose a novel dual encoder, affinity-score based decoder topology that achieves additional improvements to performance. Experiments on an internal dataset, MQ2007, and AwA2 show our approach outperforms existing alternative approaches by 21%, 8%, and 4% respectively on accuracy and NDCG metrics. On Mini-ImageNet dataset our approach achieves comparable results with Prototypical Networks. Empirical evaluations demonstrate that our approach provides a unified and effective framework.