NAAS: Neural Accelerator Architecture Search
This work addresses the need for efficient and specialized neural accelerator design for AI hardware developers, offering a data-driven approach that improves over previous incremental methods.
The paper tackles the problem of automatically designing neural accelerator architectures by proposing NAAS, which holistically searches neural network architecture, accelerator architecture, and compiler mapping in one optimization loop, resulting in a 4.4x reduction in EDP and 2.7% accuracy improvement on ImageNet compared to human-designed Eyeriss.
Data-driven, automatic design space exploration of neural accelerator architecture is desirable for specialization and productivity. Previous frameworks focus on sizing the numerical architectural hyper-parameters while neglect searching the PE connectivities and compiler mappings. To tackle this challenge, we propose Neural Accelerator Architecture Search (NAAS) which holistically searches the neural network architecture, accelerator architecture, and compiler mapping in one optimization loop. NAAS composes highly matched architectures together with efficient mapping. As a data-driven approach, NAAS rivals the human design Eyeriss by 4.4x EDP reduction with 2.7% accuracy improvement on ImageNet under the same computation resource, and offers 1.4x to 3.5x EDP reduction than only sizing the architectural hyper-parameters.