Continuous Ant-Based Neural Topology Search
This work provides an easier-to-use and competitive neural architecture search method for researchers and practitioners working on time series prediction, particularly in power systems.
This paper introduces Continuous Ant-based Neural Topology Search (CANTS), a novel ant colony optimization algorithm for neural architecture search. CANTS automates the design of ANNs of any size by operating in a continuous search space, addressing a limitation of many existing NAS algorithms. It achieves improved or competitive results on three real-world time series prediction problems in power systems, while requiring half the number of user-specified hyper-parameters compared to state-of-the-art algorithms.
This work introduces a novel, nature-inspired neural architecture search (NAS) algorithm based on ant colony optimization, Continuous Ant-based Neural Topology Search (CANTS), which utilizes synthetic ants that move over a continuous search space based on the density and distribution of pheromones, is strongly inspired by how ants move in the real world. The paths taken by the ant agents through the search space are utilized to construct artificial neural networks (ANNs). This continuous search space allows CANTS to automate the design of ANNs of any size, removing a key limitation inherent to many current NAS algorithms that must operate within structures with a size predetermined by the user. CANTS employs a distributed asynchronous strategy which allows it to scale to large-scale high performance computing resources, works with a variety of recurrent memory cell structures, and makes use of a communal weight sharing strategy to reduce training time. The proposed procedure is evaluated on three real-world, time series prediction problems in the field of power systems and compared to two state-of-the-art algorithms. Results show that CANTS is able to provide improved or competitive results on all of these problems, while also being easier to use, requiring half the number of user-specified hyper-parameters.