Stress Testing of Meta-learning Approaches for Few-shot Learning
This work identifies limitations in meta-learning methods for few-shot learning, which is important for researchers and practitioners in resource-constrained AI applications, but it is incremental as it builds on existing approaches.
The paper stress-tests meta-learning approaches for few-shot learning by evaluating their performance against increasing task complexity, finding that initialization-based methods like MAML degrade quickly, while optimization-based methods like MetaLSTM perform significantly better and show higher transferability.
Meta-learning (ML) has emerged as a promising learning method under resource constraints such as few-shot learning. ML approaches typically propose a methodology to learn generalizable models. In this work-in-progress paper, we put the recent ML approaches to a stress test to discover their limitations. Precisely, we measure the performance of ML approaches for few-shot learning against increasing task complexity. Our results show a quick degradation in the performance of initialization strategies for ML (MAML, TAML, and MetaSGD), while surprisingly, approaches that use an optimization strategy (MetaLSTM) perform significantly better. We further demonstrate the effectiveness of an optimization strategy for ML (MetaLSTM++) trained in a MAML manner over a pure optimization strategy. Our experiments also show that the optimization strategies for ML achieve higher transferability from simple to complex tasks.