MAML is a Noisy Contrastive Learner in Classification
This provides a clearer understanding of MAML's mechanism for meta-learning researchers, though it is incremental as it builds on existing MAML analysis.
The paper reveals that Model-agnostic meta-learning (MAML) functions as a supervised contrastive learner in classification, and identifies an interference issue from random initialization and cross-task interactions, proposing a 'zeroing trick' that improves performance on mini-ImageNet and Omniglot datasets.
Model-agnostic meta-learning (MAML) is one of the most popular and widely adopted meta-learning algorithms, achieving remarkable success in various learning problems. Yet, with the unique design of nested inner-loop and outer-loop updates, which govern the task-specific and meta-model-centric learning, respectively, the underlying learning objective of MAML remains implicit and thus impedes a more straightforward understanding of it. In this paper, we provide a new perspective of the working mechanism of MAML. We discover that MAML is analogous to a meta-learner using a supervised contrastive objective. The query features are pulled towards the support features of the same class and against those of different classes. Such contrastiveness is experimentally verified via an analysis based on the cosine similarity. Moreover, we reveal that vanilla MAML has an undesirable interference term originating from the random initialization and the cross-task interaction. We thus propose a simple but effective technique, zeroing trick, to alleviate the interference. Extensive experiments are conducted on both mini-ImageNet and Omniglot datasets to demonstrate the consistent improvement brought by our proposed method, validating its effectiveness.