Evaluating Disentanglement of Structured Representations
This work provides a more systematic and unified evaluation framework for researchers working on object-centric generative models, addressing limitations in prior visual metrics of object separation.
The authors introduce a new metric for evaluating disentanglement in structured latent representations, specifically for object-centric generative models. This metric allows for the evaluation of object separation, disentanglement of object properties within slots, and disentanglement of intrinsic/extrinsic properties, and is theoretically shown to offer stronger guarantees than previous metrics.
We introduce the first metric for evaluating disentanglement at individual hierarchy levels of a structured latent representation. Applied to object-centric generative models, this offers a systematic, unified approach to evaluating (i) object separation between latent slots (ii) disentanglement of object properties inside individual slots (iii) disentanglement of intrinsic and extrinsic object properties. We theoretically show that for structured representations, our framework gives stronger guarantees of selecting a good model than previous disentanglement metrics. Experimentally, we demonstrate that viewing object compositionality as a disentanglement problem addresses several issues with prior visual metrics of object separation. As a core technical component, we present the first representation probing algorithm handling slot permutation invariance.