Unsupervised Image Representation Learning with Deep Latent Particles
This work addresses the challenge of unsupervised representation learning for visual scenes with multiple dynamic objects, offering a novel approach that could benefit computer vision applications.
The paper tackles the problem of disentangling object position from appearance in visual data by proposing Deep Latent Particles (DLP), a representation using low-dimensional latent particles with spatial locations and features, and demonstrates its utility in unsupervised keypoint detection, image manipulation, and video prediction for multi-object scenes.
We propose a new representation of visual data that disentangles object position from appearance. Our method, termed Deep Latent Particles (DLP), decomposes the visual input into low-dimensional latent ``particles'', where each particle is described by its spatial location and features of its surrounding region. To drive learning of such representations, we follow a VAE-based approach and introduce a prior for particle positions based on a spatial-softmax architecture, and a modification of the evidence lower bound loss inspired by the Chamfer distance between particles. We demonstrate that our DLP representations are useful for downstream tasks such as unsupervised keypoint (KP) detection, image manipulation, and video prediction for scenes composed of multiple dynamic objects. In addition, we show that our probabilistic interpretation of the problem naturally provides uncertainty estimates for particle locations, which can be used for model selection, among other tasks. Videos and code are available: https://taldatech.github.io/deep-latent-particles-web/