Optimizing Memory Efficiency for Convolution Kernels on Kepler GPUs
This work addresses memory bandwidth challenges for convolution operations on GPUs, which is crucial for accelerating deep learning applications, but it is incremental as it builds on existing GPU optimization techniques.
The paper tackled the problem of optimizing memory efficiency for convolution kernels on Kepler GPUs by developing two kernels with optimized memory access patterns, achieving 5.16X and 35.5% average performance improvements over cuDNN for special and general cases, respectively.
Convolution is a fundamental operation in many applications, such as computer vision, natural language processing, image processing, etc. Recent successes of convolutional neural networks in various deep learning applications put even higher demand on fast convolution. The high computation throughput and memory bandwidth of graphics processing units (GPUs) make GPUs a natural choice for accelerating convolution operations. However, maximally exploiting the available memory bandwidth of GPUs for convolution is a challenging task. This paper introduces a general model to address the mismatch between the memory bank width of GPUs and computation data width of threads. Based on this model, we develop two convolution kernels, one for the general case and the other for a special case with one input channel. By carefully optimizing memory access patterns and computation patterns, we design a communication-optimized kernel for the special case and a communication-reduced kernel for the general case. Experimental data based on implementations on Kepler GPUs show that our kernels achieve 5.16X and 35.5% average performance improvement over the latest cuDNN library, for the special case and the general case, respectively.