Learning to Learn with Indispensable Connections
This work addresses computational and memory overhead in meta-learning for few-shot classification, representing an incremental advancement over existing methods like MAML.
The paper tackles the problem of over-parameterization and inefficiency in meta-learning by proposing Meta-LTH, a method that uses indispensable connections based on the lottery ticket hypothesis to improve few-shot learning, resulting in a 2% accuracy improvement on the Omniglot dataset.
Meta-learning aims to solve unseen tasks with few labelled instances. Nevertheless, despite its effectiveness for quick learning in existing optimization-based methods, it has several flaws. Inconsequential connections are frequently seen during meta-training, which results in an over-parameterized neural network. Because of this, meta-testing observes unnecessary computations and extra memory overhead. To overcome such flaws. We propose a novel meta-learning method called Meta-LTH that includes indispensible (necessary) connections. We applied the lottery ticket hypothesis technique known as magnitude pruning to generate these crucial connections that can effectively solve few-shot learning problem. We aim to perform two things: (a) to find a sub-network capable of more adaptive meta-learning and (b) to learn new low-level features of unseen tasks and recombine those features with the already learned features during the meta-test phase. Experimental results show that our proposed Met-LTH method outperformed existing first-order MAML algorithm for three different classification datasets. Our method improves the classification accuracy by approximately 2% (20-way 1-shot task setting) for omniglot dataset.