Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
This work addresses the challenge of inconsistent disentanglement measurement for researchers in generative modeling, though it appears incremental as it provides an alternative method rather than a breakthrough.
The authors tackled the problem of measuring disentanglement in deep generative models by developing a method that quantifies disentanglement using topological similarity of conditional submanifolds, requiring only the generative model itself. They empirically evaluated state-of-the-art models across multiple datasets and found that their method ranks models similarly to existing methods.
Learning disentangled representations is regarded as a fundamental task for improving the generalization, robustness, and interpretability of generative models. However, measuring disentanglement has been challenging and inconsistent, often dependent on an ad-hoc external model or specific to a certain dataset. To address this, we present a method for quantifying disentanglement that only uses the generative model, by measuring the topological similarity of conditional submanifolds in the learned representation. This method showcases both unsupervised and supervised variants. To illustrate the effectiveness and applicability of our method, we empirically evaluate several state-of-the-art models across multiple datasets. We find that our method ranks models similarly to existing methods. We make ourcode publicly available at https://github.com/stanfordmlgroup/disentanglement.