Prune and Replace NAS
This work addresses the efficiency and scalability of NAS for researchers and practitioners, offering a method that is incremental in improving search space exploration.
The paper tackles the problem of Neural Architecture Search (NAS) by introducing PR-DARTS, which discovers strong network configurations in a much larger search space and within a single day, achieving test errors of 2.51% on CIFAR-10 and 15.53% on CIFAR-100.
While recent NAS algorithms are thousands of times faster than the pioneering works, it is often overlooked that they use fewer candidate operations, resulting in a significantly smaller search space. We present PR-DARTS, a NAS algorithm that discovers strong network configurations in a much larger search space and a single day. A small candidate operation pool is used, from which candidates are progressively pruned and replaced with better performing ones. Experiments on CIFAR-10 and CIFAR-100 achieve 2.51% and 15.53% test error, respectively, despite searching in a space where each cell has 150 times as many possible configurations than in the DARTS baseline. Code is available at https://github.com/cogsys-tuebingen/prdarts