Learning Pose Image Manifolds Using Geometry-Preserving GANs and Elasticae
This addresses the problem of generating realistic image sequences for novel object rotations in computer vision, representing an incremental improvement over existing GAN and VAE techniques.
The paper tackles learning pose image manifolds for 3D objects with limited data by proposing a DNN approach that combines a geometry-preserving GAN and elastica-based interpolation, achieving superior performance in generating smooth rotations compared to state-of-the-art methods.
This paper investigates the challenge of learning image manifolds, specifically pose manifolds, of 3D objects using limited training data. It proposes a DNN approach to manifold learning and for predicting images of objects for novel, continuous 3D rotations. The approach uses two distinct concepts: (1) Geometric Style-GAN (Geom-SGAN), which maps images to low-dimensional latent representations and maintains the (first-order) manifold geometry. That is, it seeks to preserve the pairwise distances between base points and their tangent spaces, and (2) uses Euler's elastica to smoothly interpolate between directed points (points + tangent directions) in the low-dimensional latent space. When mapped back to the larger image space, the resulting interpolations resemble videos of rotating objects. Extensive experiments establish the superiority of this framework in learning paths on rotation manifolds, both visually and quantitatively, relative to state-of-the-art GANs and VAEs.