Konstantinos Margaritis

George Kyriakides, Konstantinos Margaritis

NORD (Neural Operations Research & Development) is an open source distributed deep learning architectural research framework, based on PyTorch, MPI and Horovod. It aims to make research of deep architectures easier for experts of different domains, in order to accelerate the process of finding better architectures, as well as study the best architectures generated for different datasets. Although currently under heavy development, the framework aims to allow the easy implementation of different design and optimization method families (optimization algorithms, meta-heuristics, reinforcement learning etc.) as well as the fair comparison between them. Furthermore, due to the computational resources required in order to optimize and evaluate network architectures, it leverage the use of distributed computing, while aiming to minimize the researcher's overhead required to implement it. Moreover, it strives to make the creation of architectures more intuitive, by implementing network descriptors, allowing to separately define the architecture's nodes and connections. In this paper, we present the framework's current state of development, while presenting its basic concepts, providing simple examples as well as their experimental results.

Aristeidis Christoforidis, George Kyriakides, Konstantinos Margaritis

In this work, we propose a novel evolutionary algorithm for neural architecture search, applicable to global search spaces. The algorithm's architectural representation organizes the topology in multiple hierarchical modules, while the design process exploits this representation, in order to explore the search space. We also employ a curation system, which promotes the utilization of well performing sub-structures to subsequent generations. We apply our method to Fashion-MNIST and NAS-Bench101, achieving accuracies of $93.2\%$ and $94.8\%$ respectively in a relatively small number of generations.

George Kyriakides, Konstantinos Margaritis

Neural Architecture Search (NAS) is a research field concerned with utilizing optimization algorithms to design optimal neural network architectures. There are many approaches concerning the architectural search spaces, optimization algorithms, as well as candidate architecture evaluation methods. As the field is growing at a continuously increasing pace, it is difficult for a beginner to discern between major, as well as emerging directions the field has followed. In this work, we provide an introduction to the basic concepts of NAS for convolutional networks, along with the major advances in search spaces, algorithms and evaluation techniques.