Interpolated SelectionConv for Spherical Images and Surfaces
This work addresses the challenge of processing spherical images and surfaces for applications like computer vision and graphics, representing an incremental improvement by adapting existing methods to new data types.
The authors tackled the problem of applying convolutional neural networks to spherical images and surfaces by introducing a graph-based framework that uses interpolated SelectionConv, enabling the use of existing 2D CNNs and weights without relying on specific sampling strategies. They demonstrated effectiveness in style transfer and segmentation for spheres and stylization for 3D meshes, with a thorough ablation study on spherical sampling strategies.
We present a new and general framework for convolutional neural network operations on spherical (or omnidirectional) images. Our approach represents the surface as a graph of connected points that doesn't rely on a particular sampling strategy. Additionally, by using an interpolated version of SelectionConv, we can operate on the sphere while using existing 2D CNNs and their weights. Since our method leverages existing graph implementations, it is also fast and can be fine-tuned efficiently. Our method is also general enough to be applied to any surface type, even those that are topologically non-simple. We demonstrate the effectiveness of our technique on the tasks of style transfer and segmentation for spheres as well as stylization for 3D meshes. We provide a thorough ablation study of the performance of various spherical sampling strategies.