Investigating Image Manifolds of 3D Objects: Learning, Shape Analysis, and Comparisons
This work provides insights into computer vision by analyzing image manifolds, which can simplify tasks and predict performance, but it is incremental as it builds on classical manifold learning with a new geometric perspective.
The paper investigates the low-dimensional manifolds formed by images of 3D objects under rotation, finding they are nonlinear and smooth, and uses shape analysis to cluster objects by manifold shape, with objects from the same classes often grouped together.
Despite high-dimensionality of images, the sets of images of 3D objects have long been hypothesized to form low-dimensional manifolds. What is the nature of such manifolds? How do they differ across objects and object classes? Answering these questions can provide key insights in explaining and advancing success of machine learning algorithms in computer vision. This paper investigates dual tasks -- learning and analyzing shapes of image manifolds -- by revisiting a classical problem of manifold learning but from a novel geometrical perspective. It uses geometry-preserving transformations to map the pose image manifolds, sets of images formed by rotating 3D objects, to low-dimensional latent spaces. The pose manifolds of different objects in latent spaces are found to be nonlinear, smooth manifolds. The paper then compares shapes of these manifolds for different objects using Kendall's shape analysis, modulo rigid motions and global scaling, and clusters objects according to these shape metrics. Interestingly, pose manifolds for objects from the same classes are frequently clustered together. The geometries of image manifolds can be exploited to simplify vision and image processing tasks, to predict performances, and to provide insights into learning methods.