MANAS: Multi-Agent Neural Architecture Search
This addresses the practical use of NAS for machine learning practitioners by improving efficiency, though it appears incremental as it builds on existing NAS frameworks.
The paper tackles the efficiency bottleneck in Neural Architecture Search (NAS) by framing it as a multi-agent problem, achieving reduced memory requirements (1/8th of state-of-the-art) and performances above more computationally expensive methods.
The Neural Architecture Search (NAS) problem is typically formulated as a graph search problem where the goal is to learn the optimal operations over edges in order to maximise a graph-level global objective. Due to the large architecture parameter space, efficiency is a key bottleneck preventing NAS from its practical use. In this paper, we address the issue by framing NAS as a multi-agent problem where agents control a subset of the network and coordinate to reach optimal architectures. We provide two distinct lightweight implementations, with reduced memory requirements (1/8th of state-of-the-art), and performances above those of much more computationally expensive methods. Theoretically, we demonstrate vanishing regrets of the form O(sqrt(T)), with T being the total number of rounds. Finally, aware that random search is an, often ignored, effective baseline we perform additional experiments on 3 alternative datasets and 2 network configurations, and achieve favourable results in comparison.