Kolmogorov-Arnold Fourier Networks
This work addresses computational and representational bottlenecks in interpretable neural networks for high-dimensional tasks, offering a practical solution with broad applicability.
The paper tackles the parameter explosion and high-frequency feature capture challenges in Kolmogorov-Arnold based interpretable networks (KAN) by proposing the Kolmogorov-Arnold-Fourier Network (KAF), which integrates trainable Random Fourier Features and a hybrid GELU-Fourier activation to balance efficiency and spectral representation, achieving superior performance across vision, NLP, audio, and differential equation-solving tasks.
Although Kolmogorov-Arnold based interpretable networks (KAN) have strong theoretical expressiveness, they face significant parameter explosion and high-frequency feature capture challenges in high-dimensional tasks. To address this issue, we propose the Kolmogorov-Arnold-Fourier Network (KAF), which effectively integrates trainable Random Fourier Features (RFF) and a novel hybrid GELU-Fourier activation mechanism to balance parameter efficiency and spectral representation capabilities. Our key technical contributions include: (1) merging KAN's dual-matrix structure through matrix association properties to substantially reduce parameters; (2) introducing learnable RFF initialization strategies to eliminate spectral distortion in high-dimensional approximation tasks; (3) implementing an adaptive hybrid activation function that progressively enhances frequency representation during the training process. Comprehensive experiments demonstrate the superiority of our KAF across various domains including vision, NLP, audio processing, and differential equation-solving tasks, effectively combining theoretical interpretability with practical utility and computational efficiency.