One-Shot Neural Architecture Search via Compressive Sensing
This addresses the meta-learning problem of NAS for researchers and practitioners by offering a more efficient method, though it appears incremental as it builds on parameter-sharing ideas.
The paper tackles the challenge of reducing Neural Architecture Search (NAS) running time by proposing an iterative technique inspired by algorithms for learning low-degree sparse Boolean functions, achieving competitive performance on DARTs and NAS-Bench-201 benchmarks.
Neural Architecture Search remains a very challenging meta-learning problem. Several recent techniques based on parameter-sharing idea have focused on reducing the NAS running time by leveraging proxy models, leading to architectures with competitive performance compared to those with hand-crafted designs. In this paper, we propose an iterative technique for NAS, inspired by algorithms for learning low-degree sparse Boolean functions. We validate our approach on the DARTs search space (Liu et al., 2018b) and NAS-Bench-201 (Yang et al., 2020). In addition, we provide theoretical analysis via upper bounds on the number of validation error measurements needed for reliable learning, and include ablation studies to further in-depth understanding of our technique.