How to Accelerate Capsule Convolutions in Capsule Networks
This work addresses a performance bottleneck for researchers and practitioners using Capsule Networks, though it is incremental as it focuses on optimizing a specific component.
The paper tackled the inefficiency of capsule convolutions in Capsule Networks, which are slow due to incompatibility with existing deep learning frameworks, and achieved a 4X acceleration by developing two CUDA-based schemes.
How to improve the efficiency of routing procedures in CapsNets has been studied a lot. However, the efficiency of capsule convolutions has largely been neglected. Capsule convolution, which uses capsules rather than neurons as the basic computation unit, makes it incompatible with current deep learning frameworks' optimization solution. As a result, capsule convolutions are usually very slow with these frameworks. We observe that capsule convolutions can be considered as the operations of `multiplication of multiple small matrics' plus tensor-based combination. Based on this observation, we develop two acceleration schemes with CUDA APIs and test them on a custom CapsNet. The result shows that our solution achieves a 4X acceleration.