LSS-SKAN: Efficient Kolmogorov-Arnold Networks based on Single-Parameterized Function
This work addresses computational bottlenecks for researchers using KANs, though it is incremental as it builds on existing KAN frameworks.
The paper tackles the inefficiency of Kolmogorov-Arnold Networks (KANs) by proposing a new variant called LSS-SKAN, which uses a single-parameter basis function based on the Efficient KAN Expansion Principle, resulting in superior accuracy and faster execution speed compared to other KAN variants on the MNIST dataset.
The recently proposed Kolmogorov-Arnold Networks (KAN) networks have attracted increasing attention due to their advantage of high visualizability compared to MLP. In this paper, based on a series of small-scale experiments, we proposed the Efficient KAN Expansion Principle (EKE Principle): allocating parameters to expand network scale, rather than employing more complex basis functions, leads to more efficient performance improvements in KANs. Based on this principle, we proposed a superior KAN termed SKAN, where the basis function utilizes only a single learnable parameter. We then evaluated various single-parameterized functions for constructing SKANs, with LShifted Softplus-based SKANs (LSS-SKANs) demonstrating superior accuracy. Subsequently, extensive experiments were performed, comparing LSS-SKAN with other KAN variants on the MNIST dataset. In the final accuracy tests, LSS-SKAN exhibited superior performance on the MNIST dataset compared to all tested pure KAN variants. Regarding execution speed, LSS-SKAN outperformed all compared popular KAN variants. Our experimental codes are available at https://github.com/chikkkit/LSS-SKAN and SKAN's Python library (for quick construction of SKAN in python) codes are available at https://github.com/chikkkit/SKAN .