Multi-Exit Kolmogorov-Arnold Networks: enhancing accuracy and parsimony
This addresses a fundamental challenge in machine learning for scientific discovery by providing a practical way to balance high performance and interpretability, though it is incremental as it builds on existing KANs.
The paper tackles the problem of determining optimal depth for Kolmogorov-Arnold Networks (KANs) by introducing multi-exit KANs, which enable accurate predictions at multiple depths and consistently outperform standard versions on synthetic and real-world datasets, often using earlier, simpler exits to achieve parsimony without sacrificing accuracy.
Kolmogorov-Arnold Networks (KANs) uniquely combine high accuracy with interpretability, making them valuable for scientific modeling. However, it is unclear a priori how deep a network needs to be for any given task, and deeper KANs can be difficult to optimize and interpret. Here we introduce multi-exit KANs, where each layer includes its own prediction branch, enabling the network to make accurate predictions at multiple depths simultaneously. This architecture provides deep supervision that improves training while discovering the right level of model complexity for each task. Multi-exit KANs consistently outperform standard, single-exit versions on synthetic functions, dynamical systems, and real-world datasets. Remarkably, the best predictions often come from earlier, simpler exits, revealing that these networks naturally identify smaller, more parsimonious and interpretable models without sacrificing accuracy. To automate this discovery, we develop a differentiable "learning-to-exit" algorithm that balances contributions from exits during training. Our approach offers scientists a practical way to achieve both high performance and interpretability, addressing a fundamental challenge in machine learning for scientific discovery.