Neural Architecture Search Over a Graph Search Space
This work addresses a bottleneck in NAS for researchers and practitioners by enabling more flexible architecture design, though it is incremental as it builds on existing NAS methods.
The paper tackled the problem of limited expressiveness in Neural Architecture Search (NAS) search spaces by proposing a graph-based representation, and demonstrated improved sample efficiency in image classification experiments.
Neural Architecture Search (NAS) enabled the discovery of state-of-the-art architectures in many domains. However, the success of NAS depends on the definition of the search space. Current search spaces are defined as a static sequence of decisions and a set of available actions for each decision. Each possible sequence of actions defines an architecture. We propose a more expressive class of search space: directed graphs. In our formalism, each decision is a vertex and each action is an edge. This allows us to model iterative and branching architecture design decisions. We demonstrate in simulation, and on image classification experiments, basic iterative and branching search structures, and show that the graph representation improves sample efficiency.