Intrinsic and Triangulation-Agnostic Attention: A Simple and Powerful Approach for Learning on Meshes
For researchers in geometry processing and 3D deep learning, this provides a simple yet effective method that outperforms existing mesh-based and point cloud transformers on several benchmarks.
This work adapts the attention mechanism for triangle meshes by incorporating intrinsicality and triangulation-agnosticism, achieving state-of-the-art results across multiple geometry-processing tasks, including deformation prediction, dense correspondence, and feature descriptor prediction.
This work proposes an adaptation of the attention mechanism for triangle meshes. The core observation is that endowing the attention mechanism with critical properties for learning over meshes -- intrinsicality and triangulation-agnosticism -- enables it to attain state-of-the-art results over several learning-based tasks in geometry-processing. The above is achieved by modifying the attention mechanism from the bottom up based on simple principles from geometry-processing. Namely, the quantities used within attention -- queries, keys and values -- are created by an intrinsic, triangulation-agnostic network, and treated as discretizations of continuous functions. From that, we devise an appropriate attention mechanism that operates over triangle meshes through standard FEM discretization of the resulting integrals of the above functions. Surprisingly, as far as we know, this straightforward approach has not been utilized for learning over meshes. Experiments show our method exceeds current state of the art, including both mesh-based architectures as well as point cloud transformers. Namely, we show significant improvements on several common benchmarks and tasks -- predicting canonical high-frequency signals; predicting deformations; computing dense correspondences, both between full shapes and partial ones; and predicting feature descriptors.