Optimizing Routerless Network-on-Chip Designs: An Innovative Learning-Based Framework
This work addresses the challenge of efficient network-on-chip design for computer architecture, offering significant performance improvements but is incremental as it builds on existing deep reinforcement learning methods.
The paper tackles the problem of designing routerless networks-on-chip by proposing a deep reinforcement learning framework that learns near-optimal loop placements, achieving a 3.25x increase in throughput and 1.6x reduction in packet latency compared to conventional mesh designs.
Machine learning applied to architecture design presents a promising opportunity with broad applications. Recent deep reinforcement learning (DRL) techniques, in particular, enable efficient exploration in vast design spaces where conventional design strategies may be inadequate. This paper proposes a novel deep reinforcement framework, taking routerless networks-on-chip (NoC) as an evaluation case study. The new framework successfully resolves problems with prior design approaches being either unreliable due to random searches or inflexible due to severe design space restrictions. The framework learns (near-)optimal loop placement for routerless NoCs with various design constraints. A deep neural network is developed using parallel threads that efficiently explore the immense routerless NoC design space with a Monte Carlo search tree. Experimental results show that, compared with conventional mesh, the proposed deep reinforcement learning (DRL) routerless design achieves a 3.25x increase in throughput, 1.6x reduction in packet latency, and 5x reduction in power. Compared with the state-of-the-art routerless NoC, DRL achieves a 1.47x increase in throughput, 1.18x reduction in packet latency, and 1.14x reduction in average hop count albeit with slightly more power overhead.