Visual Reasoning of Feature Attribution with Deep Recurrent Neural Networks
This work addresses the need for interpretability in deep RNNs for data scientists, though it is incremental as it builds on existing visualization and attention methods.
The paper tackles the problem of understanding feature attribution in deep recurrent neural networks (RNNs) by developing a visual analytics approach that reveals attention mechanisms, temporal positions, and variable attributions, demonstrating its utility on real-world datasets to guide modeling processes.
Deep Recurrent Neural Network (RNN) has gained popularity in many sequence classification tasks. Beyond predicting a correct class for each data instance, data scientists also want to understand what differentiating factors in the data have contributed to the classification during the learning process. We present a visual analytics approach to facilitate this task by revealing the RNN attention for all data instances, their temporal positions in the sequences, and the attribution of variables at each value level. We demonstrate with real-world datasets that our approach can help data scientists to understand such dynamics in deep RNNs from the training results, hence guiding their modeling process.