Understanding Recurrent Neural State Using Memory Signatures
This provides a tool for researchers to analyze memory in recurrent neural networks, but it is incremental as it builds on existing visualization methods.
The paper tackles the challenge of interpreting recurrent state in LSTMs/GRUs by introducing a visualization technique that decodes prior input history, revealing insights into memory behavior in grapheme sequence models.
We demonstrate a network visualization technique to analyze the recurrent state inside the LSTMs/GRUs used commonly in language and acoustic models. Interpreting intermediate state and network activations inside end-to-end models remains an open challenge. Our method allows users to understand exactly how much and what history is encoded inside recurrent state in grapheme sequence models. Our procedure trains multiple decoders that predict prior input history. Compiling results from these decoders, a user can obtain a signature of the recurrent kernel that characterizes its memory behavior. We demonstrate this method's usefulness in revealing information divergence in the bases of recurrent factorized kernels, visualizing the character-level differences between the memory of n-gram and recurrent language models, and extracting knowledge of history encoded in the layers of grapheme-based end-to-end ASR networks.