An Open-Source HW-SW Co-Development Framework Enabling Efficient Multi-Accelerator Systems
This addresses the challenge of balancing performance and ease of use in multi-accelerator systems for AI developers, though it appears incremental as it builds on existing integration strategies with a novel hybrid-coupling scheme.
The paper tackled the problem of inefficient data movement and compatibility issues in heterogeneous accelerator systems for AI workloads by presenting SNAX, an open-source HW-SW framework, which achieved over 10x improvement in neural network performance and over 90% accelerator utilization in a low-power SoC.
Heterogeneous accelerator-centric compute clusters are emerging as efficient solutions for diverse AI workloads. However, current integration strategies often compromise data movement efficiency and encounter compatibility issues in hardware and software. This prevents a unified approach that balances performance and ease of use. To this end, we present SNAX, an open-source integrated HW-SW framework enabling efficient multi-accelerator platforms through a novel hybrid-coupling scheme, consisting of loosely coupled asynchronous control and tightly coupled data access. SNAX brings reusable hardware modules designed to enhance compute accelerator utilization, and its customizable MLIR-based compiler to automate key system management tasks, jointly enabling rapid development and deployment of customized multi-accelerator compute clusters. Through extensive experimentation, we demonstrate SNAX's efficiency and flexibility in a low-power heterogeneous SoC. Accelerators can easily be integrated and programmed to achieve > 10x improvement in neural network performance compared to other accelerator systems while maintaining accelerator utilization of > 90% in full system operation.