Neural Architecture Search with Random Labels
This work addresses the challenge of reducing label dependency in NAS, potentially making it more efficient, though it appears incremental as it builds on existing NAS paradigms.
The paper tackles the problem of neural architecture search (NAS) by introducing a method that uses random labels during search, based on the ease-of-convergence hypothesis, and achieves comparable or better results than state-of-the-art NAS methods on datasets like NAS-Bench-201 and ImageNet.
In this paper, we investigate a new variant of neural architecture search (NAS) paradigm -- searching with random labels (RLNAS). The task sounds counter-intuitive for most existing NAS algorithms since random label provides few information on the performance of each candidate architecture. Instead, we propose a novel NAS framework based on ease-of-convergence hypothesis, which requires only random labels during searching. The algorithm involves two steps: first, we train a SuperNet using random labels; second, from the SuperNet we extract the sub-network whose weights change most significantly during the training. Extensive experiments are evaluated on multiple datasets (e.g. NAS-Bench-201 and ImageNet) and multiple search spaces (e.g. DARTS-like and MobileNet-like). Very surprisingly, RLNAS achieves comparable or even better results compared with state-of-the-art NAS methods such as PC-DARTS, Single Path One-Shot, even though the counterparts utilize full ground truth labels for searching. We hope our finding could inspire new understandings on the essential of NAS.