Compositionality Unlocks Deep Interpretable Models
This work addresses the problem of interpretability in deep learning for researchers and practitioners, offering a method that is incremental by building on existing tensor network and neural network techniques.
The authors tackled the challenge of creating interpretable deep learning models by introducing $\chi$-net, which combines tensor networks with neural networks to maintain accuracy while enabling interpretability and compression, achieving equal accuracy to baseline models and revealing linear low-rank structure in SVHN.
We propose $χ$-net, an intrinsically interpretable architecture combining the compositional multilinear structure of tensor networks with the expressivity and efficiency of deep neural networks. $χ$-nets retain equal accuracy compared to their baseline counterparts. Our novel, efficient diagonalisation algorithm, ODT, reveals linear low-rank structure in a multilayer SVHN model. We leverage this toward formal weight-based interpretability and model compression.