Momentum Capsule Networks
This work addresses efficiency issues in capsule networks for computer vision, but it is incremental as it adapts reversible methods from Momentum ResNets.
The authors tackled the high computation and memory requirements of baseline capsule networks by proposing Momentum Capsule Network (MoCapsNet), which beats baseline accuracy on MNIST, SVHN, CIFAR-10, and CIFAR-100 while using considerably less memory.
Capsule networks are a class of neural networks that achieved promising results on many computer vision tasks. However, baseline capsule networks have failed to reach state-of-the-art results on more complex datasets due to the high computation and memory requirements. We tackle this problem by proposing a new network architecture, called Momentum Capsule Network (MoCapsNet). MoCapsNets are inspired by Momentum ResNets, a type of network that applies reversible residual building blocks. Reversible networks allow for recalculating activations of the forward pass in the backpropagation algorithm, so those memory requirements can be drastically reduced. In this paper, we provide a framework on how invertible residual building blocks can be applied to capsule networks. We will show that MoCapsNet beats the accuracy of baseline capsule networks on MNIST, SVHN, CIFAR-10 and CIFAR-100 while using considerably less memory. The source code is available on https://github.com/moejoe95/MoCapsNet.