Topological Data Analysis of Neural Network Layer Representations
This is an incremental study on understanding neural network internal representations for researchers in machine learning and topology.
The paper studied how topological features are preserved in neural network layer representations using topological data analysis, finding that early layers approximated homeomorphisms while deeper layers altered topology, with noise hampering feature computation but bijective activation functions showing longer persistence.
This paper is a cursory study on how topological features are preserved within the internal representations of neural network layers. Using techniques from topological data analysis, namely persistent homology, the topological features of a simple feedforward neural network's layer representations of a modified torus with a Klein bottle-like twist were computed. The network appeared to approximate homeomorphisms in early layers, before significantly changing the topology of the data in deeper layers. The resulting noise hampered the ability of persistent homology to compute these features, however similar topological features seemed to persist longer in a network with a bijective activation function.