MARS: Exploiting Multi-Level Parallelism for DNN Workloads on Adaptive Multi-Accelerator Systems
This addresses performance optimization for DNN workloads in data centers and cloud platforms, representing an incremental improvement over existing mapping techniques.
The paper tackles the problem of selecting accelerators and mapping strategies for deep neural networks on multi-accelerator systems, achieving an average 32.2% latency reduction for typical workloads and 59.4% reduction for heterogeneous models compared to state-of-the-art methods.
Along with the fast evolution of deep neural networks, the hardware system is also developing rapidly. As a promising solution achieving high scalability and low manufacturing cost, multi-accelerator systems widely exist in data centers, cloud platforms, and SoCs. Thus, a challenging problem arises in multi-accelerator systems: selecting a proper combination of accelerators from available designs and searching for efficient DNN mapping strategies. To this end, we propose MARS, a novel mapping framework that can perform computation-aware accelerator selection, and apply communication-aware sharding strategies to maximize parallelism. Experimental results show that MARS can achieve 32.2% latency reduction on average for typical DNN workloads compared to the baseline, and 59.4% latency reduction on heterogeneous models compared to the corresponding state-of-the-art method.