Transferring disentangled representations: bridging the gap between synthetic and real images
This addresses the problem of limited ground truth labels for real images in machine learning, though it is incremental in nature.
The paper tackled the challenge of applying disentangled representation learning to real images by leveraging synthetic data, showing that transferring representations from synthetic to real data is possible and effective.
Developing meaningful and efficient representations that separate the fundamental structure of the data generation mechanism is crucial in representation learning. However, Disentangled Representation Learning has not fully shown its potential on real images, because of correlated generative factors, their resolution and limited access to ground truth labels. Specifically on the latter, we investigate the possibility of leveraging synthetic data to learn general-purpose disentangled representations applicable to real data, discussing the effect of fine-tuning and what properties of disentanglement are preserved after the transfer. We provide an extensive empirical study to address these issues. In addition, we propose a new interpretable intervention-based metric, to measure the quality of factors encoding in the representation. Our results indicate that some level of disentanglement, transferring a representation from synthetic to real data, is possible and effective.