Adaptive Gradient-Based Meta-Learning Methods
This work addresses the need for adaptive and theoretically grounded meta-learning algorithms, with incremental improvements for researchers and practitioners in machine learning.
The paper tackles the problem of designing practical meta-learning methods by integrating task-similarity formalizations with online convex optimization, resulting in improved meta-test-time performance on few-shot and federated learning benchmarks.
We build a theoretical framework for designing and understanding practical meta-learning methods that integrates sophisticated formalizations of task-similarity with the extensive literature on online convex optimization and sequential prediction algorithms. Our approach enables the task-similarity to be learned adaptively, provides sharper transfer-risk bounds in the setting of statistical learning-to-learn, and leads to straightforward derivations of average-case regret bounds for efficient algorithms in settings where the task-environment changes dynamically or the tasks share a certain geometric structure. We use our theory to modify several popular meta-learning algorithms and improve their meta-test-time performance on standard problems in few-shot learning and federated learning.