Fast Dynamic Routing Based on Weighted Kernel Density Estimation
This work addresses the computational bottleneck limiting widespread use of capsule networks, offering incremental improvements for efficient deep learning applications.
The paper tackles the high computational cost of dynamic routing in capsule networks by proposing two fast routing methods based on weighted kernel density estimation, which improve time efficiency by nearly 40% with minimal performance loss. The models achieve competitive results with other leading methods on multiple benchmarks using a hybrid network architecture for 64x64 pixel inputs.
Capsules as well as dynamic routing between them are most recently proposed structures for deep neural networks. A capsule groups data into vectors or matrices as poses rather than conventional scalars to represent specific properties of target instance. Besides of pose, a capsule should be attached with a probability (often denoted as activation) for its presence. The dynamic routing helps capsules achieve more generalization capacity with many fewer model parameters. However, the bottleneck that prevents widespread applications of capsule is the expense of computation during routing. To address this problem, we generalize existing routing methods within the framework of weighted kernel density estimation, and propose two fast routing methods with different optimization strategies. Our methods prompt the time efficiency of routing by nearly 40\% with negligible performance degradation. By stacking a hybrid of convolutional layers and capsule layers, we construct a network architecture to handle inputs at a resolution of $64\times{64}$ pixels. The proposed models achieve a parallel performance with other leading methods in multiple benchmarks.