An Analytical Approach to Compute the Exact Preimage of Feed-Forward Neural Networks
This addresses the black-box nature of neural networks for researchers and practitioners, offering a tool to enhance interpretability, though it is incremental as it focuses on specific activation types.
The paper tackles the problem of understanding neural network decisions by computing the exact preimage for feed-forward networks with linear or piecewise linear activations, providing an analytical method that returns the entire preimage rather than a single solution.
Neural networks are a convenient way to automatically fit functions that are too complex to be described by hand. The downside of this approach is that it leads to build a black-box without understanding what happened inside. Finding the preimage would help to better understand how and why such neural networks had given such outputs. Because most of the neural networks are noninjective function, it is often impossible to compute it entirely only by a numerical way. The point of this study is to give a method to compute the exact preimage of any Feed-Forward Neural Network with linear or piecewise linear activation functions for hidden layers. In contrast to other methods, this one is not returning a unique solution for a unique output but returns analytically the entire and exact preimage.