Softly Symbolifying Kolmogorov-Arnold Networks
This addresses the challenge of interpretability in machine learning for researchers and practitioners, though it is incremental by building on existing KANs.
The paper tackled the problem of trained activations in Kolmogorov-Arnold Networks lacking symbolic fidelity by proposing Softly Symbolified Kolmogorov-Arnold Networks (S2KAN), which integrate symbolic primitives with differentiable sparsification, achieving competitive or superior accuracy with substantially smaller models on symbolic benchmarks, dynamical systems forecasting, and real-world tasks.
Kolmogorov-Arnold Networks (KANs) offer a promising path toward interpretable machine learning: their learnable activations can be studied individually, while collectively fitting complex data accurately. In practice, however, trained activations often lack symbolic fidelity, learning pathological decompositions with no meaningful correspondence to interpretable forms. We propose Softly Symbolified Kolmogorov-Arnold Networks (S2KAN), which integrate symbolic primitives directly into training. Each activation draws from a dictionary of symbolic and dense terms, with learnable gates that sparsify the representation. Crucially, this sparsification is differentiable, enabling end-to-end optimization, and is guided by a principled Minimum Description Length objective. When symbolic terms suffice, S2KAN discovers interpretable forms; when they do not, it gracefully degrades to dense splines. We demonstrate competitive or superior accuracy with substantially smaller models across symbolic benchmarks, dynamical systems forecasting, and real-world prediction tasks, and observe evidence of emergent self-sparsification even without regularization pressure.