Disentanglement by means of action-induced representations
This work addresses the fundamental problem of nonlinear independent component analysis in representation learning for AI, offering a novel approach to disentanglement with potential applications in physical systems and beyond.
The paper tackles the challenge of learning interpretable, disentangled representations in variational autoencoders by introducing action-induced representations (AIRs), which provably disentangle degrees of freedom based on action dependence, and a variational AIR architecture (VAIR) that achieves this where standard VAEs fail.
Learning interpretable representations with variational autoencoders (VAEs) is a major goal of representation learning. The main challenge lies in obtaining disentangled representations, where each latent dimension corresponds to a distinct generative factor. This difficulty is fundamentally tied to the inability to perform nonlinear independent component analysis. Here, we introduce the framework of action-induced representations (AIRs) which models representations of physical systems given experiments (or actions) that can be performed on them. We show that, in this framework, we can provably disentangle degrees of freedom w.r.t. their action dependence. We further introduce a variational AIR architecture (VAIR) that can extract AIRs and therefore achieve provable disentanglement where standard VAEs fail. Beyond state representation, VAIR also captures the action dependence of the underlying generative factors, directly linking experiments to the degrees of freedom they influence.