Advances in MetaDL: AAAI 2021 challenge and workshop
This work addresses the need for standardized benchmarks in metalearning for researchers, but it is incremental as it builds on existing methods without introducing new paradigms.
The paper organized the AAAI 2021 MetaDL challenge and workshop to advance metalearning with deep learning, focusing on few-shot image classification under computational constraints, where winning methods used fine-tuned CNN backbones and achieved competitive results in a uniform evaluation.
To stimulate advances in metalearning using deep learning techniques (MetaDL), we organized in 2021 a challenge and an associated workshop. This paper presents the design of the challenge and its results, and summarizes presentations made at the workshop. The challenge focused on few-shot learning classification tasks of small images. Participants' code submissions were run in a uniform manner, under tight computational constraints. This put pressure on solution designs to use existing architecture backbones and/or pre-trained networks. Winning methods featured various classifiers trained on top of the second last layer of popular CNN backbones, fined-tuned on the meta-training data (not necessarily in an episodic manner), then trained on the labeled support and tested on the unlabeled query sets of the meta-test data.