Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning
This work addresses a practical limitation in few-shot learning for scenarios with data privacy or resource constraints, though it is incremental as it builds on existing MAML frameworks.
The paper tackles the problem of adapting pretrained MAML checkpoints for new few-shot learning tasks without access to the original meta-training data, proposing a method that combines adversarial training and uncertainty-based stepsize adaptation to outperform vanilla MAML on same-domain and cross-domain benchmarks.
Model-agnostic meta-learning (MAML) is a popular method for few-shot learning but assumes that we have access to the meta-training set. In practice, training on the meta-training set may not always be an option due to data privacy concerns, intellectual property issues, or merely lack of computing resources. In this paper, we consider the novel problem of repurposing pretrained MAML checkpoints to solve new few-shot classification tasks. Because of the potential distribution mismatch, the original MAML steps may no longer be optimal. Therefore we propose an alternative meta-testing procedure and combine MAML gradient steps with adversarial training and uncertainty-based stepsize adaptation. Our method outperforms "vanilla" MAML on same-domain and cross-domains benchmarks using both SGD and Adam optimizers and shows improved robustness to the choice of base stepsize.