Graph Routing between Capsules
This work addresses a key bottleneck in capsule networks for text classification, offering incremental improvements in accuracy across multiple datasets.
The paper tackles the problem of learning intra-relations between capsules in the same layer for better semantic understanding in text data, introducing a new capsule network with graph routing that achieves accuracy improvements of up to 1.01% compared to state-of-the-art methods on five text classification datasets.
Routing methods in capsule networks often learn a hierarchical relationship for capsules in successive layers, but the intra-relation between capsules in the same layer is less studied, while this intra-relation is a key factor for the semantic understanding in text data. Therefore, in this paper, we introduce a new capsule network with graph routing to learn both relationships, where capsules in each layer are treated as the nodes of a graph. We investigate strategies to yield adjacency and degree matrix with three different distances from a layer of capsules, and propose the graph routing mechanism between those capsules. We validate our approach on five text classification datasets, and our findings suggest that the approach combining bottom-up routing and top-down attention performs the best. Such an approach demonstrates generalization capability across datasets. Compared to the state-of-the-art routing methods, the improvements in accuracy in the five datasets we used were 0.82, 0.39, 0.07, 1.01, and 0.02, respectively.