Im2Mesh GAN: Accurate 3D Hand Mesh Recovery from a Single RGB Image
It addresses the problem of accurate 3D hand reconstruction for computer vision applications, but it is incremental as it builds on existing GAN and mesh learning techniques.
This work tackles hand mesh recovery from a single RGB image by learning the mesh directly without parametric models, using a new GAN called Im2Mesh GAN that captures topological relationships and 3D features, and it outperforms state-of-the-art methods.
This work addresses hand mesh recovery from a single RGB image. In contrast to most of the existing approaches where the parametric hand models are employed as the prior, we show that the hand mesh can be learned directly from the input image. We propose a new type of GAN called Im2Mesh GAN to learn the mesh through end-to-end adversarial training. By interpreting the mesh as a graph, our model is able to capture the topological relationship among the mesh vertices. We also introduce a 3D surface descriptor into the GAN architecture to further capture the 3D features associated. We experiment two approaches where one can reap the benefits of coupled groundtruth data availability of images and the corresponding meshes, while the other combats the more challenging problem of mesh estimations without the corresponding groundtruth. Through extensive evaluations we demonstrate that the proposed method outperforms the state-of-the-art.