Dissecting Deep Neural Networks
This work addresses the need for better interpretability in safety-critical applications, though it is incremental as it builds on prior research on piecewise affine properties.
The paper tackles the problem of understanding deep neural networks by developing an algorithm to compute explicit piecewise affine representations for fully connected networks with ReLU activations, moving beyond just counting linear regions.
In exchange for large quantities of data and processing power, deep neural networks have yielded models that provide state of the art predication capabilities in many fields. However, a lack of strong guarantees on their behaviour have raised concerns over their use in safety-critical applications. A first step to understanding these networks is to develop alternate representations that allow for further analysis. It has been shown that neural networks with piecewise affine activation functions are themselves piecewise affine, with their domains consisting of a vast number of linear regions. So far, the research on this topic has focused on counting the number of linear regions, rather than obtaining explicit piecewise affine representations. This work presents a novel algorithm that can compute the piecewise affine form of any fully connected neural network with rectified linear unit activations.