MeshWalker: Deep Mesh Understanding by Random Walks
This addresses the challenge of 3D shape analysis in computer graphics, where large mesh datasets are scarce, by enabling effective learning from limited data.
The paper tackles the problem of representing 3D triangular meshes for deep learning by proposing MeshWalker, which uses random walks along the mesh surface fed into an RNN, achieving state-of-the-art results in shape classification and semantic segmentation with high efficiency using few examples.
Most attempts to represent 3D shapes for deep learning have focused on volumetric grids, multi-view images and point clouds. In this paper we look at the most popular representation of 3D shapes in computer graphics - a triangular mesh - and ask how it can be utilized within deep learning. The few attempts to answer this question propose to adapt convolutions & pooling to suit Convolutional Neural Networks (CNNs). This paper proposes a very different approach, termed MeshWalker, to learn the shape directly from a given mesh. The key idea is to represent the mesh by random walks along the surface, which "explore" the mesh's geometry and topology. Each walk is organized as a list of vertices, which in some manner imposes regularity on the mesh. The walk is fed into a Recurrent Neural Network (RNN) that "remembers" the history of the walk. We show that our approach achieves state-of-the-art results for two fundamental shape analysis tasks: shape classification and semantic segmentation. Furthermore, even a very small number of examples suffices for learning. This is highly important, since large datasets of meshes are difficult to acquire.