HMCNAS: Neural Architecture Search using Hidden Markov Chains and Bayesian Optimization
This work addresses the need for more generalized and less biased NAS methods for researchers and practitioners in machine learning, though it appears incremental as it builds on existing NAS techniques.
The paper tackles the problem of human bias and assumptions in Neural Architecture Search (NAS) by proposing HMCNAS, which autonomously generates complex search spaces and uses an evolutionary algorithm with Bayesian optimization to create competitive CNNs without human-defined parameters, achieving competitive architectures in a short time.
Neural Architecture Search has achieved state-of-the-art performance in a variety of tasks, out-performing human-designed networks. However, many assumptions, that require human definition, related with the problems being solved or the models generated are still needed: final model architectures, number of layers to be sampled, forced operations, small search spaces, which ultimately contributes to having models with higher performances at the cost of inducing bias into the system. In this paper, we propose HMCNAS, which is composed of two novel components: i) a method that leverages information about human-designed models to autonomously generate a complex search space, and ii) an Evolutionary Algorithm with Bayesian Optimization that is capable of generating competitive CNNs from scratch, without relying on human-defined parameters or small search spaces. The experimental results show that the proposed approach results in competitive architectures obtained in a very short time. HMCNAS provides a step towards generalizing NAS, by providing a way to create competitive models, without requiring any human knowledge about the specific task.