A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators
This work addresses the need for domain-specific hardware optimization in datacenters, offering a practical solution for improving efficiency in deep learning inference, though it is incremental in building on existing accelerator search methods.
The paper tackles the problem of designing efficient deep learning inference accelerators for datacenter workloads by proposing FAST, a full-stack search framework that optimizes hardware and software components, resulting in accelerators that improve Perf/TDP by up to 3.7x compared to TPU-v3.
The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7x on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4x on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments.