Unwrapping All ReLU Networks
This work offers a theoretical framework for interpreting neural networks, which could benefit researchers and practitioners in machine learning by enhancing model transparency and explainability, though it appears incremental as it builds on existing decomposition theory.
The paper extends the theory of decomposing deep ReLU networks into linear models to Graph Neural Networks, tensor convolutional networks, and networks with multiplicative interactions, and provides proofs linking neural networks to interpretable models like Multivariate Decision trees and logical theories, enabling cheap and exact SHAP value computation, validated through experiments on Graph Neural Networks.
Deep ReLU Networks can be decomposed into a collection of linear models, each defined in a region of a partition of the input space. This paper provides three results extending this theory. First, we extend this linear decompositions to Graph Neural networks and tensor convolutional networks, as well as networks with multiplicative interactions. Second, we provide proofs that neural networks can be understood as interpretable models such as Multivariate Decision trees and logical theories. Finally, we show how this model leads to computing cheap and exact SHAP values. We validate the theory through experiments with on Graph Neural Networks.