SkelEx and BoundEx: Natural Visualization of ReLU Neural Networks
This work addresses the problem of understanding learned functions in ReLU NNs for researchers and practitioners, offering incremental improvements in visualization methods.
The paper tackles the interpretability of ReLU Neural Networks by introducing SkelEx to extract skeletons of membership functions and BoundEx to analytically extract decision boundaries, providing natural visualization tools for low-dimensional data.
Despite their limited interpretability, weights and biases are still the most popular encoding of the functions learned by ReLU Neural Networks (ReLU NNs). That is why we introduce SkelEx, an algorithm to extract a skeleton of the membership functions learned by ReLU NNs, making those functions easier to interpret and analyze. To the best of our knowledge, this is the first work that considers linear regions from the perspective of critical points. As a natural follow-up, we also introduce BoundEx, which is the first analytical method known to us to extract the decision boundary from the realization of a ReLU NN. Both of those methods introduce very natural visualization tool for ReLU NNs trained on low-dimensional data.