XB-MAML: Learning Expandable Basis Parameters for Effective Meta-Learning with Wide Task Coverage
This addresses the challenge of adapting meta-learning to diverse unseen tasks, but it is incremental as it builds on prior multi-initialization methods.
The paper tackles the problem of meta-learning across a wide range of tasks by introducing XB-MAML, which learns expandable basis parameters that are linearly combined for task-specific initializations, and it surpasses existing works on multi-domain benchmarks.
Meta-learning, which pursues an effective initialization model, has emerged as a promising approach to handling unseen tasks. However, a limitation remains to be evident when a meta-learner tries to encompass a wide range of task distribution, e.g., learning across distinctive datasets or domains. Recently, a group of works has attempted to employ multiple model initializations to cover widely-ranging tasks, but they are limited in adaptively expanding initializations. We introduce XB-MAML, which learns expandable basis parameters, where they are linearly combined to form an effective initialization to a given task. XB-MAML observes the discrepancy between the vector space spanned by the basis and fine-tuned parameters to decide whether to expand the basis. Our method surpasses the existing works in the multi-domain meta-learning benchmarks and opens up new chances of meta-learning for obtaining the diverse inductive bias that can be combined to stretch toward the effective initialization for diverse unseen tasks.