Modular Adaptation for Cross-Domain Few-Shot Learning
This work addresses the challenge of efficiently adapting models to new tasks with limited data, which is incremental as it builds on existing adaptation methods by combining them modularly.
The paper tackles the problem of adapting pre-trained representations for cross-domain few-shot learning by proposing a modular adaptation method that dynamically selects and sequences multiple state-of-the-art adaptation modules based on the downstream task, resulting in a 3.1% improvement in 5-shot classification accuracy over baselines.
Adapting pre-trained representations has become the go-to recipe for learning new downstream tasks with limited examples. While literature has demonstrated great successes via representation learning, in this work, we show that substantial performance improvement of downstream tasks can also be achieved by appropriate designs of the adaptation process. Specifically, we propose a modular adaptation method that selectively performs multiple state-of-the-art (SOTA) adaptation methods in sequence. As different downstream tasks may require different types of adaptation, our modular adaptation enables the dynamic configuration of the most suitable modules based on the downstream task. Moreover, as an extension to existing cross-domain 5-way k-shot benchmarks (e.g., miniImageNet -> CUB), we create a new high-way (~100) k-shot benchmark with data from 10 different datasets. This benchmark provides a diverse set of domains and allows the use of stronger representations learned from ImageNet. Experimental results show that by customizing adaptation process towards downstream tasks, our modular adaptation pipeline (MAP) improves 3.1% in 5-shot classification accuracy over baselines of finetuning and Prototypical Networks.