Disentanglement Analysis with Partial Information Decomposition
This work addresses the challenge of accurately measuring disentanglement in machine learning representations, which is crucial for interpretability and control in AI systems, though it is incremental as it builds on existing metrics.
The authors tackled the problem of detecting complex entanglement patterns in multivariate representations, proposing a new metric based on Partial Information Decomposition that correctly identifies entanglement involving multiple variables, as validated through experiments on variational autoencoders.
We propose a framework to analyze how multivariate representations disentangle ground-truth generative factors. A quantitative analysis of disentanglement has been based on metrics designed to compare how one variable explains each generative factor. Current metrics, however, may fail to detect entanglement that involves more than two variables, e.g., representations that duplicate and rotate generative factors in high dimensional spaces. In this work, we establish a framework to analyze information sharing in a multivariate representation with Partial Information Decomposition and propose a new disentanglement metric. This framework enables us to understand disentanglement in terms of uniqueness, redundancy, and synergy. We develop an experimental protocol to assess how increasingly entangled representations are evaluated with each metric and confirm that the proposed metric correctly responds to entanglement. Through experiments on variational autoencoders, we find that models with similar disentanglement scores have a variety of characteristics in entanglement, for each of which a distinct strategy may be required to obtain a disentangled representation.