Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraints
This addresses the problem of learning disentangled representations without labels for researchers in machine learning, offering a domain-general approach that is incremental over existing unsupervised methods.
The authors tackled unsupervised disentanglement of content and style from sequences by proposing V3, a method based on variance-invariance constraints, which achieved strong disentanglement performance across domains like music audio and images, with superior out-of-distribution generalization in few-shot adaptation compared to supervised methods.
We contribute an unsupervised method that effectively learns disentangled content and style representations from sequences of observations. Unlike most disentanglement algorithms that rely on domain-specific labels or knowledge, our method is based on the insight of domain-general statistical differences between content and style -- content varies more among different fragments within a sample but maintains an invariant vocabulary across data samples, whereas style remains relatively invariant within a sample but exhibits more significant variation across different samples. We integrate such inductive bias into an encoder-decoder architecture and name our method after V3 (variance-versus-invariance). Experimental results show that V3 generalizes across multiple domains and modalities, successfully learning disentangled content and style representations, such as pitch and timbre from music audio, digit and color from images of hand-written digits, and action and character appearance from simple animations. V3 demonstrates strong disentanglement performance compared to existing unsupervised methods, along with superior out-of-distribution generalization under few-shot adaptation compared to supervised counterparts. Lastly, symbolic-level interpretability emerges in the learned content codebook, forging a near one-to-one alignment between machine representation and human knowledge.