Set-based Meta-Interpolation for Few-Task Meta-Learning
This addresses the bottleneck of requiring many meta-training tasks for generalization in real-world problems with few tasks, offering a domain-agnostic solution that is incremental over prior task augmentation methods.
The paper tackles the problem of few-task meta-learning by proposing Meta-Interpolation, a domain-agnostic task augmentation method that uses neural set functions and bilevel optimization to densify the meta-training task distribution, and it consistently outperforms baselines across eight datasets in domains like image classification and text classification.
Meta-learning approaches enable machine learning systems to adapt to new tasks given few examples by leveraging knowledge from related tasks. However, a large number of meta-training tasks are still required for generalization to unseen tasks during meta-testing, which introduces a critical bottleneck for real-world problems that come with only few tasks, due to various reasons including the difficulty and cost of constructing tasks. Recently, several task augmentation methods have been proposed to tackle this issue using domain-specific knowledge to design augmentation techniques to densify the meta-training task distribution. However, such reliance on domain-specific knowledge renders these methods inapplicable to other domains. While Manifold Mixup based task augmentation methods are domain-agnostic, we empirically find them ineffective on non-image domains. To tackle these limitations, we propose a novel domain-agnostic task augmentation method, Meta-Interpolation, which utilizes expressive neural set functions to densify the meta-training task distribution using bilevel optimization. We empirically validate the efficacy of Meta-Interpolation on eight datasets spanning across various domains such as image classification, molecule property prediction, text classification and speech recognition. Experimentally, we show that Meta-Interpolation consistently outperforms all the relevant baselines. Theoretically, we prove that task interpolation with the set function regularizes the meta-learner to improve generalization.