Accelerating Gradient-based Meta Learner
This work addresses the time and iteration inefficiency in meta learning, which is a problem for researchers and practitioners seeking faster adaptation to new tasks, though it appears incremental.
The paper tackles the slow convergence of gradient-based meta learners by proposing acceleration techniques, achieving a 3.73x speedup on an RNN optimizer-based meta learner while improving model accuracy through task clustering.
Meta Learning has been in focus in recent years due to the meta-learner model's ability to adapt well and generalize to new tasks, thus, reducing both the time and data requirements for learning. However, a major drawback of meta learner is that, to reach to a state from where learning new tasks becomes feasible with less data, it requires a large number of iterations and a lot of time. We address this issue by proposing various acceleration techniques to speed up meta learning algorithms such as MAML (Model Agnostic Meta Learning). We present 3.73X acceleration on a well known RNN optimizer based meta learner proposed in literature [11]. We introduce a novel method of training tasks in clusters, which not only accelerates the meta learning process but also improves model accuracy performance. Keywords: Meta learning, RNN optimizer, AGI, Performance optimization