Group-disentangled Representation Learning with Weakly-Supervised Regularization
This work addresses the problem of interpretable representation learning for AI researchers, offering an incremental improvement over existing weakly-supervised techniques.
The paper tackles the challenge of learning group-disentangled representations with weak supervision by proposing GroupVAE, a method using KL divergence regularization to enforce consistency, which significantly improves group disentanglement and enhances performance on downstream tasks like fair classification and 3D shape tasks.
Learning interpretable and human-controllable representations that uncover factors of variation in data remains an ongoing key challenge in representation learning. We investigate learning group-disentangled representations for groups of factors with weak supervision. Existing techniques to address this challenge merely constrain the approximate posterior by averaging over observations of a shared group. As a result, observations with a common set of variations are encoded to distinct latent representations, reducing their capacity to disentangle and generalize to downstream tasks. In contrast to previous works, we propose GroupVAE, a simple yet effective Kullback-Leibler (KL) divergence-based regularization across shared latent representations to enforce consistent and disentangled representations. We conduct a thorough evaluation and demonstrate that our GroupVAE significantly improves group disentanglement. Further, we demonstrate that learning group-disentangled representations improve upon downstream tasks, including fair classification and 3D shape-related tasks such as reconstruction, classification, and transfer learning, and is competitive to supervised methods.