Metric as Transform: Exploring beyond Affine Transform for Interpretable Neural Network
This work addresses interpretability issues for neural network users, but it is incremental as it builds on existing affine transform methods with limited performance gains.
The paper tackles the problem of interpretability in neural networks by generalizing dot product neurons to metrics and beyond, finding that metrics as transform perform similarly to affine transforms in MLPs and CNNs while offering better interpretability in some cases, such as developing an interpretable local dictionary network to understand and reject adversarial examples.
Artificial Neural Networks of varying architectures are generally paired with affine transformation at the core. However, we find dot product neurons with global influence less interpretable as compared to local influence of euclidean distance (as used in Radial Basis Function Network). In this work, we explore the generalization of dot product neurons to $l^p$-norm, metrics, and beyond. We find that metrics as transform performs similarly to affine transform when used in MultiLayer Perceptron or Convolutional Neural Network. Moreover, we explore various properties of Metrics, compare it with Affine, and present multiple cases where metrics seem to provide better interpretability. We develop an interpretable local dictionary based Neural Networks and use it to understand and reject adversarial examples.