Learning Linear Symmetries in Data Using Moment Matching
This work addresses the challenge of automatically discovering symmetries from data, which could reduce reliance on expert knowledge in machine learning, though it appears incremental as it builds on existing symmetry concepts with theoretical and empirical extensions.
The paper tackles the problem of learning linear symmetries in data distribution in an unsupervised or semi-supervised manner, showing that it is as hard as the graph automorphism problem in the worst case but becomes tractable under specific conditions like unique eigenvalues and orthogonal symmetries, with methods developed and compared for selecting eigenvectors with eigenvalue -1.
It is common in machine learning and statistics to use symmetries derived from expert knowledge to simplify problems or improve performance, using methods like data augmentation or penalties. In this paper we consider the unsupervised and semi-supervised problems of learning such symmetries in a distribution directly from data in a model-free fashion. We show that in the worst case this problem is as difficult as the graph automorphism problem. However, if we restrict to the case where the covariance matrix has unique eigenvalues, then the eigenvectors will also be eigenvectors of the symmetry transformation. If we further restrict to finding orthogonal symmetries, then the eigenvalues will be either be 1 or -1, and the problem reduces to determining which eigenvectors are which. We develop and compare theoretically and empirically the effectiveness of different methods of selecting which eigenvectors should have eigenvalue -1 in the symmetry transformation, and discuss how to extend this approach to non-orthogonal cases where we have labels