Qonvolution: Towards Learning High-Frequency Signals with Queried Convolution
This addresses a key problem in computer vision and graphics for researchers and practitioners, offering incremental improvements over existing techniques like Fourier encodings.
The paper tackles the challenge of learning high-frequency signals in computer vision and graphics by introducing Queried-Convolutions (Qonvolutions), which convolve low-frequency signals with queries to enhance performance, achieving state-of-the-art results in tasks like novel view synthesis and outperforming radiance field models on image quality.
Accurately learning high-frequency signals is a challenge in computer vision and graphics, as neural networks often struggle with these signals due to spectral bias or optimization difficulties. While current techniques like Fourier encodings have made great strides in improving performance, there remains scope for improvement when presented with high-frequency information. This paper introduces Queried-Convolutions (Qonvolutions), a simple yet powerful modification using the neighborhood properties of convolution. Qonvolution convolves a low-frequency signal with queries (such as coordinates) to enhance the learning of intricate high-frequency signals. We empirically demonstrate that Qonvolutions enhance performance across a variety of high-frequency learning tasks crucial to both the computer vision and graphics communities, including 1D regression, 2D super-resolution, 2D image regression, and novel view synthesis (NVS). In particular, by combining Gaussian splatting with Qonvolutions for NVS, we showcase state-of-the-art performance on real-world complex scenes, even outperforming powerful radiance field models on image quality.