From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flows
This addresses the problem of unsupervised representation learning for researchers and practitioners, offering an incremental improvement by combining existing insights into a flexible framework.
The paper tackled the challenge of learning unsupervised representations that are semantically meaningful and stable by introducing entropy-ordered flows (EOFlows), a normalizing-flow framework that orders latent dimensions by explained entropy, enabling adaptive core-detail separation; experiments on CelebA showed it uncovers interpretable features, allowing for high compression and strong denoising.
Learning unsupervised representations that are both semantically meaningful and stable across runs remains a central challenge in modern representation learning. We introduce entropy-ordered flows (EOFlows), a normalizing-flow framework that orders latent dimensions by their explained entropy, analogously to PCA's explained variance. This ordering enables adaptive injective flows: after training, one may retain only the top C latent variables to form a compact core representation while the remaining variables capture fine-grained detail and noise, with C chosen flexibly at inference time rather than fixed during training. EOFlows build on insights from Independent Mechanism Analysis, Principal Component Flows and Manifold Entropic Metrics. We combine likelihood-based training with local Jacobian regularization and noise augmentation into a method that scales well to high-dimensional data such as images. Experiments on the CelebA dataset show that our method uncovers a rich set of semantically interpretable features, allowing for high compression and strong denoising.