Mutation is all you need
This is an incremental analysis for researchers in automated machine learning, focusing on understanding specific components of an existing NAS method.
The paper investigates the performance of the BANANAS neural architecture search method on the NAS-Bench-301 benchmark, finding that its acquisition function optimizer, which uses minimal mutation, is the key determinant of its results.
Neural architecture search (NAS) promises to make deep learning accessible to non-experts by automating architecture engineering of deep neural networks. BANANAS is one state-of-the-art NAS method that is embedded within the Bayesian optimization framework. Recent experimental findings have demonstrated the strong performance of BANANAS on the NAS-Bench-101 benchmark being determined by its path encoding and not its choice of surrogate model. We present experimental results suggesting that the performance of BANANAS on the NAS-Bench-301 benchmark is determined by its acquisition function optimizer, which minimally mutates the incumbent.