LieGG: Studying Learned Lie Group Generators
This addresses the challenge of understanding and quantifying learned symmetries in neural networks for researchers in machine learning, though it is incremental as it builds on existing symmetry learning concepts.
The paper tackles the problem of extracting and evaluating symmetries learned by neural networks from data, presenting a method to retrieve learned invariances as Lie group generators without prior knowledge. They found that the ability to learn symmetries generalizes across architectures, but quality depends on depth and parameter count.
Symmetries built into a neural network have appeared to be very beneficial for a wide range of tasks as it saves the data to learn them. We depart from the position that when symmetries are not built into a model a priori, it is advantageous for robust networks to learn symmetries directly from the data to fit a task function. In this paper, we present a method to extract symmetries learned by a neural network and to evaluate the degree to which a network is invariant to them. With our method, we are able to explicitly retrieve learned invariances in a form of the generators of corresponding Lie-groups without prior knowledge of symmetries in the data. We use the proposed method to study how symmetrical properties depend on a neural network's parameterization and configuration. We found that the ability of a network to learn symmetries generalizes over a range of architectures. However, the quality of learned symmetries depends on the depth and the number of parameters.