RACE: A Reinforcement Learning Framework for Improved Adaptive Control of NoC Channel Buffers
This work addresses adaptive buffer control for network-on-chip systems, offering significant performance and energy gains, though it is incremental as it builds on existing reversible buffer technology.
The paper tackles the problem of controlling reversible multi-function channel buffers in network-on-chip architectures to improve performance and reduce energy, achieving up to 48.9% latency reduction and 47.1% energy reduction compared to state-of-the-art policies.
Network-on-chip (NoC) architectures rely on buffers to store flits to cope with contention for router resources during packet switching. Recently, reversible multi-function channel (RMC) buffers have been proposed to simultaneously reduce power and enable adaptive NoC buffering between adjacent routers. While adaptive buffering can improve NoC performance by maximizing buffer utilization, controlling the RMC buffer allocations requires a congestion-aware, scalable, and proactive policy. In this work, we present RACE, a novel reinforcement learning (RL) framework that utilizes better awareness of network congestion and a new reward metric ("falsefulls") to help guide the RL agent towards better RMC buffer control decisions. We show that RACE reduces NoC latency by up to 48.9%, and energy consumption by up to 47.1% against state-of-the-art NoC buffer control policies.