WeLa-VAE: Learning Alternative Disentangled Representations Using Weak Labels
This work addresses the challenge of acquiring interpretable disentangled representations in machine learning with easier-to-obtain weak labels, offering a domain-specific incremental improvement over existing methods.
The paper tackles the problem of learning interpretable disentangled representations by introducing weak supervision through high-level labels, proposing WeLa-VAE as a variational inference framework that generalizes TCVAE with minimal hyperparameter addition. The result shows that WeLa-VAE successfully learns and disentangles a polar representation from weak labels of distance and angle on a Cartesian dataset, without requiring refined labels or adjustments to model parameters.
Learning disentangled representations without supervision or inductive biases, often leads to non-interpretable or undesirable representations. On the other hand, strict supervision requires detailed knowledge of the true generative factors, which is not always possible. In this paper, we consider weak supervision by means of high-level labels that are not assumed to be explicitly related to the ground truth factors. Such labels, while being easier to acquire, can also be used as inductive biases for algorithms to learn more interpretable or alternative disentangled representations. To this end, we propose WeLa-VAE, a variational inference framework where observations and labels share the same latent variables, which involves the maximization of a modified variational lower bound and total correlation regularization. Our method is a generalization of TCVAE, adding only one extra hyperparameter. We experiment on a dataset generated by Cartesian coordinates and we show that, while a TCVAE learns a factorized Cartesian representation, given weak labels of distance and angle, WeLa-VAE is able to learn and disentangle a polar representation. This is achieved without the need of refined labels or having to adjust the number of layers, the optimization parameters, or the total correlation hyperparameter.