A Local Optima Network Analysis of the Feedforward Neural Architecture Space
This provides an early indication for researchers in neural architecture search that LONs could be a viable paradigm, though it is incremental as it builds on existing fitness landscape concepts.
The study tackled the problem of analyzing the neural architecture space by applying local optima network (LON) analysis to fully enumerated feedforward networks, finding that LONs exhibit simple global structures with single global funnels in most cases, indicating their potential for architecture optimization.
This study investigates the use of local optima network (LON) analysis, a derivative of the fitness landscape of candidate solutions, to characterise and visualise the neural architecture space. The search space of feedforward neural network architectures with up to three layers, each with up to 10 neurons, is fully enumerated by evaluating trained model performance on a selection of data sets. Extracted LONs, while heterogeneous across data sets, all exhibit simple global structures, with single global funnels in all cases but one. These results yield early indication that LONs may provide a viable paradigm by which to analyse and optimise neural architectures.