Optimally Scheduling CNN Convolutions for Efficient Memory Access
This work addresses memory efficiency for embedded AI systems, offering incremental improvements in scheduling and hardware design.
The paper tackles the problem of optimizing memory bandwidth for CNN convolutions in embedded inference engines by developing an analytical model for loop-nest optimization, resulting in up to 14x memory bandwidth reduction compared to a prior accelerator.
Embedded inference engines for convolutional networks must be parsimonious in memory bandwidth and buffer sizing to meet power and cost constraints. We present an analytical memory bandwidth model for loop-nest optimization targeting architectures with application managed buffers. We applied this model to optimize the CNN convolution loop-nest. We show that our model is more accurate than previously published models. Using this model we can identify non-trivial dataflow schedules that result in lowest communication bandwidth given tight local buffering constraints. We show that optimal dataflow schedules are implementable in practice and that our model is accurate with respect to a real implementation; moreover, we introduce an accelerator architecture, named Hardware Convolution Block (HWC), which implements the optimal schedules, and we show it achieves up to 14x memory bandwidth reduction compared to a previously published accelerator with a similar memory interface, but implementing a non-optimal schedule.