P-CapsNets: a General Form of Convolutional Neural Networks
This work addresses the computational bottleneck in Capsule Networks for researchers and practitioners, offering a more efficient alternative, though it is incremental as it builds on existing CapsNet ideas.
The authors tackled the inefficiency of Capsule Networks by proposing P-CapsNets, which remove routing procedures and modify convolutional layers to reduce parameters, achieving over 99% accuracy on MNIST with only 3888 parameters and outperforming CapsNets on MNIST and CIFAR10.
We propose Pure CapsNets (P-CapsNets) which is a generation of normal CNNs structurally. Specifically, we make three modifications to current CapsNets. First, we remove routing procedures from CapsNets based on the observation that the coupling coefficients can be learned implicitly. Second, we replace the convolutional layers in CapsNets to improve efficiency. Third, we package the capsules into rank-3 tensors to further improve efficiency. The experiment shows that P-CapsNets achieve better performance than CapsNets with varied routing procedures by using significantly fewer parameters on MNIST\&CIFAR10. The high efficiency of P-CapsNets is even comparable to some deep compressing models. For example, we achieve more than 99\% percent accuracy on MNIST by using only 3888 parameters. We visualize the capsules as well as the corresponding correlation matrix to show a possible way of initializing CapsNets in the future. We also explore the adversarial robustness of P-CapsNets compared to CNNs.