Learned Hardware/Software Co-Design of Neural Accelerators
This addresses the challenge of efficiently designing specialized deep learning systems for researchers and engineers, though it is incremental as it builds on existing co-design concepts with a new optimization method.
The paper tackles the problem of hardware/software co-design for neural accelerators by proposing a constrained Bayesian optimization framework to automatically explore the joint design space, resulting in improvements such as an 18% better energy-delay product for ResNet and 40% for DQN over hand-tuned state-of-the-art systems.
The use of deep learning has grown at an exponential rate, giving rise to numerous specialized hardware and software systems for deep learning. Because the design space of deep learning software stacks and hardware accelerators is diverse and vast, prior work considers software optimizations separately from hardware architectures, effectively reducing the search space. Unfortunately, this bifurcated approach means that many profitable design points are never explored. This paper instead casts the problem as hardware/software co-design, with the goal of automatically identifying desirable points in the joint design space. The key to our solution is a new constrained Bayesian optimization framework that avoids invalid solutions by exploiting the highly constrained features of this design space, which are semi-continuous/semi-discrete. We evaluate our optimization framework by applying it to a variety of neural models, improving the energy-delay product by 18% (ResNet) and 40% (DQN) over hand-tuned state-of-the-art systems, as well as demonstrating strong results on other neural network architectures, such as MLPs and Transformers.