Multiway Multislice PHATE: Visualizing Hidden Dynamics of RNNs through Training
This provides a tool for researchers and practitioners to analyze RNN training dynamics, though it is incremental as it builds on existing visualization techniques.
The paper tackles the problem of understanding RNNs as black boxes by introducing Multiway Multislice PHATE (MM-PHATE), a method for visualizing the evolution of hidden states during training, which identifies information processing phases and preserves community structure among units.
Recurrent neural networks (RNNs) are a widely used tool for sequential data analysis, however, they are still often seen as black boxes of computation. Understanding the functional principles of these networks is critical to developing ideal model architectures and optimization strategies. Previous studies typically only emphasize the network representation post-training, overlooking their evolution process throughout training. Here, we present Multiway Multislice PHATE (MM-PHATE), a novel method for visualizing the evolution of RNNs' hidden states. MM-PHATE is a graph-based embedding using structured kernels across the multiple dimensions spanned by RNNs: time, training epoch, and units. We demonstrate on various datasets that MM-PHATE uniquely preserves hidden representation community structure among units and identifies information processing and compression phases during training. The embedding allows users to look under the hood of RNNs across training and provides an intuitive and comprehensive strategy to understanding the network's internal dynamics and draw conclusions, e.g., on why and how one model outperforms another or how a specific architecture might impact an RNN's learning ability.