Topological Signatures of ReLU Neural Network Activation Patterns
This work provides insights into neural network interpretability for researchers, though it is incremental as it builds on existing topological methods.
The paper investigates topological signatures of ReLU neural network activation patterns by analyzing polytope decompositions, finding that the Fiedler partition correlates with decision boundaries in binary classification and that homology patterns align with training loss in regression.
This paper explores the topological signatures of ReLU neural network activation patterns. We consider feedforward neural networks with ReLU activation functions and analyze the polytope decomposition of the feature space induced by the network. Mainly, we investigate how the Fiedler partition of the dual graph and show that it appears to correlate with the decision boundary -- in the case of binary classification. Additionally, we compute the homology of the cellular decomposition -- in a regression task -- to draw similar patterns in behavior between the training loss and polyhedral cell-count, as the model is trained.