Commutativity and Disentanglement from the Manifold Perspective
This work provides a theoretical foundation for disentanglement in machine learning, potentially benefiting researchers in representation learning and generative modeling, though it appears incremental by merging existing frameworks.
The paper tackles the problem of disentangling factors of variation in data by interpreting it as discovering local charts on a manifold, leading to commutativity as a key condition. It applies this framework to learning matrix exponential operators and compressing models, showing how it merges with group-theoretic and probabilistic approaches.
In this paper, we interpret disentanglement as the discovery of local charts of the data manifold and trace how this definition naturally leads to an equivalent condition for disentanglement: commutativity between factors of variation. We study the impact of this manifold framework to two classes of problems: learning matrix exponential operators and compressing data-generating models. In each problem, the manifold perspective yields interesting results about the feasibility and fruitful approaches their solutions. We also link our manifold framework to two other common disentanglement paradigms: group theoretic and probabilistic approaches to disentanglement. In each case, we show how these frameworks can be merged with our manifold perspective. Importantly, we recover commutativity as a central property in both alternative frameworks, further highlighting its importance in disentanglement.