Neural Trajectory Analysis of Recurrent Neural Network In Handwriting Synthesis
This provides incremental insights into RNN interpretability for researchers in machine learning and neuroscience, focusing on handwriting synthesis.
The study tackled the problem of understanding how recurrent neural networks (RNNs) generate realistic handwritings by analyzing neural trajectories, revealing that different writing styles are encoded in distinct subspaces and characters within styles have unique state dynamics.
Recurrent neural networks (RNNs) are capable of learning to generate highly realistic, online handwritings in a wide variety of styles from a given text sequence. Furthermore, the networks can generate handwritings in the style of a particular writer when the network states are primed with a real sequence of pen movements from the writer. However, how populations of neurons in the RNN collectively achieve such performance still remains poorly understood. To tackle this problem, we investigated learned representations in RNNs by extracting low-dimensional, neural trajectories that summarize the activity of a population of neurons in the network during individual syntheses of handwritings. The neural trajectories show that different writing styles are encoded in different subspaces inside an internal space of the network. Within each subspace, different characters of the same style are represented as different state dynamics. These results demonstrate the effectiveness of analyzing the neural trajectory for intuitive understanding of how the RNNs work.