RHNAS: Realizable Hardware and Neural Architecture Search
This addresses productivity challenges in AI by enabling efficient co-design of neural networks and hardware accelerators for researchers and engineers, though it appears incremental as it builds on existing differentiable methods.
The paper tackled the problem of non-synthesizable designs in neural network and hardware accelerator co-design by introducing RHNAS, which combines reinforcement learning and differentiable search to achieve realizable designs with 1.84x lower latency and 1.86x lower energy-delay product on ImageNet, and 2.81x lower latency and 3.30x lower EDP on CIFAR-10 compared to default hardware.
The rapidly evolving field of Artificial Intelligence necessitates automated approaches to co-design neural network architecture and neural accelerators to maximize system efficiency and address productivity challenges. To enable joint optimization of this vast space, there has been growing interest in differentiable NN-HW co-design. Fully differentiable co-design has reduced the resource requirements for discovering optimized NN-HW configurations, but fail to adapt to general hardware accelerator search spaces. This is due to the existence of non-synthesizable (invalid) designs in the search space of many hardware accelerators. To enable efficient and realizable co-design of configurable hardware accelerators with arbitrary neural network search spaces, we introduce RHNAS. RHNAS is a method that combines reinforcement learning for hardware optimization with differentiable neural architecture search. RHNAS discovers realizable NN-HW designs with 1.84x lower latency and 1.86x lower energy-delay product (EDP) on ImageNet and 2.81x lower latency and 3.30x lower EDP on CIFAR-10 over the default hardware accelerator design.