Learning to generate imaginary tasks for improving generalization in meta-learning
This addresses the issue of task distribution mismatch in meta-learning, which is crucial for applications with limited tasks, though it appears incremental as it builds on existing task augmentation methods.
The paper tackles the problem of meta-learner overfitting due to insufficient or narrow meta-training task distributions by proposing Adversarial Task Up-sampling (ATU), which generates imaginary tasks to improve generalization, resulting in marked improvements over state-of-the-art task augmentation strategies on few-shot sine regression and image classification datasets.
The success of meta-learning on existing benchmarks is predicated on the assumption that the distribution of meta-training tasks covers meta-testing tasks. Frequent violation of the assumption in applications with either insufficient tasks or a very narrow meta-training task distribution leads to memorization or learner overfitting. Recent solutions have pursued augmentation of meta-training tasks, while it is still an open question to generate both correct and sufficiently imaginary tasks. In this paper, we seek an approach that up-samples meta-training tasks from the task representation via a task up-sampling network. Besides, the resulting approach named Adversarial Task Up-sampling (ATU) suffices to generate tasks that can maximally contribute to the latest meta-learner by maximizing an adversarial loss. On few-shot sine regression and image classification datasets, we empirically validate the marked improvement of ATU over state-of-the-art task augmentation strategies in the meta-testing performance and also the quality of up-sampled tasks.