Switched linear projections for neural network interpretability
This work provides a method for improving interpretability in deep neural networks, particularly for researchers and practitioners in AI, though it appears incremental by building on existing interpretability techniques.
The paper tackles the problem of interpreting neural network activity by introducing switched linear projections to express neuron activity as a single linear projection in the input space, isolating the active subnetwork for each input instance. It also proposes examining patterns that deactivate neurons in ReLU networks, addressing a gap in existing methods.
We introduce switched linear projections for expressing the activity of a neuron in a deep neural network in terms of a single linear projection in the input space. The method works by isolating the active subnetwork, a series of linear transformations, that determine the entire computation of the network for a given input instance. With these projections we can decompose activity in any hidden layer into patterns detected in a given input instance. We also propose that in ReLU networks it is instructive and meaningful to examine patterns that deactivate the neurons in a hidden layer, something that is implicitly ignored by the existing interpretability methods tracking solely the active aspect of the network's computation.