Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks
This work addresses a theoretical gap for researchers in neural network theory by linking new KAN architectures to established ReLU networks, but it is incremental as it focuses on mathematical equivalence without new performance gains.
The paper tackles the problem of connecting Kolmogorov-Arnold Networks (KANs) with piecewise linear functions to ReLU networks by providing explicit constructions for conversion between them, establishing a formal relationship between these neural network architectures.
Kolmogorov-Arnold Networks are a new family of neural network architectures which holds promise for overcoming the curse of dimensionality and has interpretability benefits (arXiv:2404.19756). In this paper, we explore the connection between Kolmogorov Arnold Networks (KANs) with piecewise linear (univariate real) functions and ReLU networks. We provide completely explicit constructions to convert a piecewise linear KAN into a ReLU network and vice versa.