Unsupervised Part-Based Disentangling of Object Shape and Appearance
This addresses the problem of handling large intra-class variation in computer vision for applications like image synthesis and video translation, representing an incremental advance over prior unsupervised methods.
The paper tackles unsupervised disentanglement of object shape and appearance by learning consistent parts across instances, using invariance and equivariance constraints on synthetic transformations. It outperforms state-of-the-art on unsupervised keypoint prediction and matches supervised methods in shape and appearance transfer tasks.
Large intra-class variation is the result of changes in multiple object characteristics. Images, however, only show the superposition of different variable factors such as appearance or shape. Therefore, learning to disentangle and represent these different characteristics poses a great challenge, especially in the unsupervised case. Moreover, large object articulation calls for a flexible part-based model. We present an unsupervised approach for disentangling appearance and shape by learning parts consistently over all instances of a category. Our model for learning an object representation is trained by simultaneously exploiting invariance and equivariance constraints between synthetically transformed images. Since no part annotation or prior information on an object class is required, the approach is applicable to arbitrary classes. We evaluate our approach on a wide range of object categories and diverse tasks including pose prediction, disentangled image synthesis, and video-to-video translation. The approach outperforms the state-of-the-art on unsupervised keypoint prediction and compares favorably even against supervised approaches on the task of shape and appearance transfer.