Effectiveness of the Recent Advances in Capsule Networks
This is an incremental review paper that identifies gaps in capsule network research, particularly regarding squash functions, for the machine learning community.
This paper provides an overview of recent advances in capsule network architectures and routing mechanisms, while also presenting new insights on the effect of squash functions on performance.
Convolutional neural networks (CNNs) have revolutionized the field of deep neural networks. However, recent research has shown that CNNs fail to generalize under various conditions and hence the idea of capsules was introduced in 2011, though the real surge of research started from 2017. In this paper, we present an overview of the recent advances in capsule architecture and routing mechanisms. In addition, we find that the relative focus in recent literature is on modifying routing procedure or architecture as a whole but the study of other finer components, specifically, squash function is wanting. Thus, we also present some new insights regarding the effect of squash functions in performance of the capsule networks. Finally, we conclude by discussing and proposing possible opportunities in the field of capsule networks.