Rapid Model Architecture Adaption for Meta-Learning
This addresses the combinatorial search efficiency problem for few-shot learning across multiple tasks and hardware deployments, representing a novel integration rather than an incremental improvement.
The paper tackles the challenge of efficiently adapting model architectures to many tasks and hardware platforms in few-shot learning by integrating Model Agnostic Meta Learning into Network Architecture Search, resulting in a method that outperforms the best manual baseline by 5.21% in accuracy with 60% less computation on a Mini-ImageNet task.
Network Architecture Search (NAS) methods have recently gathered much attention. They design networks with better performance and use a much shorter search time compared to traditional manual tuning. Despite their efficiency in model deployments, most NAS algorithms target a single task on a fixed hardware system. However, real-life few-shot learning environments often cover a great number of tasks (T ) and deployments on a wide variety of hardware platforms (H ). The combinatorial search complexity T times H creates a fundamental search efficiency challenge if one naively applies existing NAS methods to these scenarios. To overcome this issue, we show, for the first time, how to rapidly adapt model architectures to new tasks in a many-task many-hardware few-shot learning setup by integrating Model Agnostic Meta Learning (MAML) into the NAS flow. The proposed NAS method (H-Meta-NAS) is hardware-aware and performs optimisation in the MAML framework. H-Meta-NAS shows a Pareto dominance compared to a variety of NAS and manual baselines in popular few-shot learning benchmarks with various hardware platforms and constraints. In particular, on the 5-way 1-shot Mini-ImageNet classification task, the proposed method outperforms the best manual baseline by a large margin (5.21% in accuracy) using 60% less computation.