Sparse Unsupervised Capsules Generalize Better
This addresses the issue of enabling deeper capsule networks for researchers in computer vision, though it appears incremental as it builds on existing capsule architectures.
The paper tackled the problem of unsupervised capsule networks losing equivariance and desirable qualities when trained only with reconstruction loss, which limits their depth. By introducing sparsening of latent capsule activity, they restored these qualities and improved classification accuracy on affNIST from 79% to 90%.
We show that unsupervised training of latent capsule layers using only the reconstruction loss, without masking to select the correct output class, causes a loss of equivariances and other desirable capsule qualities. This implies that supervised capsules networks can't be very deep. Unsupervised sparsening of latent capsule layer activity both restores these qualities and appears to generalize better than supervised masking, while potentially enabling deeper capsules networks. We train a sparse, unsupervised capsules network of similar geometry to Sabour et al (2017) on MNIST, and then test classification accuracy on affNIST using an SVM layer. Accuracy is improved from benchmark 79% to 90%.