Scaling Continuous Kernels with Sparse Fourier Domain Learning
This addresses a problem for researchers and practitioners in machine learning by enabling more practical adoption of continuous kernels, though it appears incremental as it builds on existing kernel methods.
The paper tackles the challenges of computational efficiency, parameter efficiency, and spectral bias in learning continuous kernel representations by proposing a novel approach using sparse learning in the Fourier domain, resulting in drastically reduced computational and memory requirements and mitigation of spectral bias.
We address three key challenges in learning continuous kernel representations: computational efficiency, parameter efficiency, and spectral bias. Continuous kernels have shown significant potential, but their practical adoption is often limited by high computational and memory demands. Additionally, these methods are prone to spectral bias, which impedes their ability to capture high-frequency details. To overcome these limitations, we propose a novel approach that leverages sparse learning in the Fourier domain. Our method enables the efficient scaling of continuous kernels, drastically reduces computational and memory requirements, and mitigates spectral bias by exploiting the Gibbs phenomenon.