Model-Agnostic Meta-Learning using Runge-Kutta Methods
This work provides an incremental improvement to meta-learning methods for researchers and practitioners seeking better few-shot learning performance.
The paper tackled the problem of improving model-agnostic meta-learning (MAML) by analyzing its optimization dynamics using Runge-Kutta methods, resulting in enhanced performance on benchmark tasks in classification, regression, and reinforcement learning.
Meta-learning has emerged as an important framework for learning new tasks from just a few examples. The success of any meta-learning model depends on (i) its fast adaptation to new tasks, as well as (ii) having a shared representation across similar tasks. Here we extend the model-agnostic meta-learning (MAML) framework introduced by Finn et al. (2017) to achieve improved performance by analyzing the temporal dynamics of the optimization procedure via the Runge-Kutta method. This method enables us to gain fine-grained control over the optimization and helps us achieve both the adaptation and representation goals across tasks. By leveraging this refined control, we demonstrate that there are multiple principled ways to update MAML and show that the classic MAML optimization is simply a special case of second-order Runge-Kutta method that mainly focuses on fast-adaptation. Experiments on benchmark classification, regression and reinforcement learning tasks show that this refined control helps attain improved results.