Neural Ensemble Search via Bayesian Sampling
This addresses the need for better performance in automated neural network design by leveraging ensembles, offering a practical improvement over existing NAS methods.
The paper tackles the problem of neural architecture search (NAS) focusing on single architectures by introducing a neural ensemble search algorithm (NESBS) that selects ensembles of diverse architectures, achieving improved performance over state-of-the-art NAS algorithms with comparable search cost.
Recently, neural architecture search (NAS) has been applied to automate the design of neural networks in real-world applications. A large number of algorithms have been developed to improve the search cost or the performance of the final selected architectures in NAS. Unfortunately, these NAS algorithms aim to select only one single well-performing architecture from their search spaces and thus have overlooked the capability of neural network ensemble (i.e., an ensemble of neural networks with diverse architectures) in achieving improved performance over a single final selected architecture. To this end, we introduce a novel neural ensemble search algorithm, called neural ensemble search via Bayesian sampling (NESBS), to effectively and efficiently select well-performing neural network ensembles from a NAS search space. In our extensive experiments, NESBS algorithm is shown to be able to achieve improved performance over state-of-the-art NAS algorithms while incurring a comparable search cost, thus indicating the superior performance of our NESBS algorithm over these NAS algorithms in practice.