NeuralArTS: Structuring Neural Architecture Search with Type Theory
This work addresses the challenge of automating and optimizing NAS search spaces for researchers and practitioners, but it appears incremental as it builds on existing NAS methods without claiming major performance gains.
The paper tackles the problem of manually designing search spaces for Neural Architecture Search (NAS) by introducing NeuralArTS, a framework that structures network operations using a type system, and demonstrates its application to convolutional layers.
Neural Architecture Search (NAS) algorithms automate the task of finding optimal deep learning architectures given an initial search space of possible operations. Developing these search spaces is usually a manual affair with pre-optimized search spaces being more efficient, rather than searching from scratch. In this paper we present a new framework called Neural Architecture Type System (NeuralArTS) that categorizes the infinite set of network operations in a structured type system. We further demonstrate how NeuralArTS can be applied to convolutional layers and propose several future directions.