On Enforcing Better Conditioned Meta-Learning for Rapid Few-Shot Adaptation
This work addresses the challenge of rapid adaptation in few-shot learning for AI systems, though it is incremental as it builds on existing meta-learning methods.
The paper tackles the problem of slow adaptation in gradient-based meta-learning by proposing a method to enforce a well-conditioned parameter space, which significantly improves initial adaptation speed without extra parameters and achieves comparable or better overall results on few-shot classification tasks.
Inspired by the concept of preconditioning, we propose a novel method to increase adaptation speed for gradient-based meta-learning methods without incurring extra parameters. We demonstrate that recasting the optimization problem to a non-linear least-squares formulation provides a principled way to actively enforce a $\textit{well-conditioned}$ parameter space for meta-learning models based on the concepts of the condition number and local curvature. Our comprehensive evaluations show that the proposed method significantly outperforms its unconstrained counterpart especially during initial adaptation steps, while achieving comparable or better overall results on several few-shot classification tasks -- creating the possibility of dynamically choosing the number of adaptation steps at inference time.