Routing Towards Discriminative Power of Class Capsules
This work addresses a specific bottleneck in capsule networks for researchers in neural network architectures, but it is incremental as it builds on existing routing schemes.
The paper tackles the suboptimal convergence of dynamic routing in capsule networks by proposing a new routing algorithm that uses a regularized quadratic programming problem to directly enhance the discriminative power of class capsules, achieving competitive classification results on MNIST, MNIST-Fashion, and CIFAR-10 datasets.
Capsule networks are recently proposed as an alternative to modern neural network architectures. Neurons are replaced with capsule units that represent specific features or entities with normalized vectors or matrices. The activation of lower layer capsules affects the behavior of the following capsules via routing links that are constructed during training via certain routing algorithms. We discuss the routing-by-agreement scheme in dynamic routing algorithm which, in certain cases, leads the networks away from optimality. To obtain better and faster convergence, we propose a routing algorithm that incorporates a regularized quadratic programming problem which can be solved efficiently. Particularly, the proposed routing algorithm targets directly on the discriminative power of class capsules making the correct decision on input instances. We conduct experiments on MNIST, MNIST-Fashion, and CIFAR-10 and show competitive classification results compared to existing capsule networks.