GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability
This addresses the issue of parameter identifiability for researchers and practitioners in machine learning, though it appears incremental as it builds on existing concepts of symmetries and equivalence.
The paper tackles the problem of visualizing symmetries and functional equivalences in neural network parameters, proposing GENNI to efficiently identify and visualize equivalence classes, enabling better exploration of identifiability questions with applications to optimization and generalizability.
We propose an efficient algorithm to visualise symmetries in neural networks. Typically, models are defined with respect to a parameter space, where non-equal parameters can produce the same input-output map. Our proposed method, GENNI, allows us to efficiently identify parameters that are functionally equivalent and then visualise the subspace of the resulting equivalence class. By doing so, we are now able to better explore questions surrounding identifiability, with applications to optimisation and generalizability, for commonly used or newly developed neural network architectures.