Operationalizing Quantized Disentanglement
This addresses the problem of unsupervised disentanglement for researchers in representation learning, though it appears incremental as it operationalizes existing theory.
The paper tackled the challenge of translating theoretical identifiability of quantized factors into a practical unsupervised disentanglement method by encouraging axis-aligned discontinuities in density estimates, showing that their method Cliff outperforms baselines on all disentanglement benchmarks.
Recent theoretical work established the unsupervised identifiability of quantized factors under any diffeomorphism. The theory assumes that quantization thresholds correspond to axis-aligned discontinuities in the probability density of the latent factors. By constraining a learned map to have a density with axis-aligned discontinuities, we can recover the quantization of the factors. However, translating this high-level principle into an effective practical criterion remains challenging, especially under nonlinear maps. Here, we develop a criterion for unsupervised disentanglement by encouraging axis-aligned discontinuities. Discontinuities manifest as sharp changes in the estimated density of factors and form what we call cliffs. Following the definition of independent discontinuities from the theory, we encourage the location of the cliffs along a factor to be independent of the values of the other factors. We show that our method, Cliff, outperforms the baselines on all disentanglement benchmarks, demonstrating its effectiveness in unsupervised disentanglement.