Variational Kolmogorov-Arnold Network
This addresses a bottleneck for researchers and practitioners using KANs as an alternative to MLPs, though it is incremental as it builds directly on existing KAN theory.
The paper tackles the problem of ad-hoc hyperparameter selection in Kolmogorov-Arnold Networks (KANs) by proposing InfinityKAN, which adaptively learns an infinite number of bases for univariate functions during training, resulting in a method that extends KANs' applicability.
Kolmogorov Arnold Networks (KANs) are an emerging architecture for building machine learning models. KANs are based on the theoretical foundation of the Kolmogorov-Arnold Theorem and its expansions, which provide an exact representation of a multi-variate continuous bounded function as the composition of a limited number of univariate continuous functions. While such theoretical results are powerful, their use as a representation learning alternative to a multi-layer perceptron (MLP) hinges on the ad-hoc choice of the number of bases modeling each of the univariate functions. In this work, we show how to address this problem by adaptively learning a potentially infinite number of bases for each univariate function during training. We therefore model the problem as a variational inference optimization problem. Our proposal, called InfinityKAN, which uses backpropagation, extends the potential applicability of KANs by treating an important hyperparameter as part of the learning process.