Transformation Coding: Simple Objectives for Equivariant Representations
This work addresses the need for flexible equivariant representation learning in machine learning, offering a non-generative method that is incremental in its approach.
The paper tackles the problem of learning equivariant deep representations without architectural constraints by introducing transformation coding objectives for various Lie groups, achieving improved disentanglement and evaluating performance on downstream tasks like reinforcement learning.
We present a simple non-generative approach to deep representation learning that seeks equivariant deep embedding through simple objectives. In contrast to existing equivariant networks, our transformation coding approach does not constrain the choice of the feed-forward layer or the architecture and allows for an unknown group action on the input space. We introduce several such transformation coding objectives for different Lie groups such as the Euclidean, Orthogonal and the Unitary groups. When using product groups, the representation is decomposed and disentangled. We show that the presence of additional information on different transformations improves disentanglement in transformation coding. We evaluate the representations learnt by transformation coding both qualitatively and quantitatively on downstream tasks, including reinforcement learning.