Explicit Construction of Approximate Kolmogorov Superpositions with C2 Smoothness
Provides a smooth, constructive approximation to Kolmogorov superpositions, potentially benefiting neural network theory and approximation of high-dimensional functions.
The authors construct an approximate Kolmogorov superposition with C2 smooth inner and outer functions that approximates α-Hölder continuous functions with error N^{-α}, where N is the number of outer summations. This overcomes the pathological behavior of exact Kolmogorov superpositions while retaining their representational essence.
We explicitly construct an approximate version of the Kolmogorov superpositions, which is composed of C2-inner and outer functions, and can approximate an arbitrary alpha Holder continuous function with accuracy of N to the power -alpha, where N denotes the number of outer summations. The inner functions are generated by applying suitable translations and dilations to a piecewise C2, strictly increasing function, while the outer functions are constructed rowwise through piecewise C2 interpolation using newly designed shape functions. This novel variant of Kolmogorov superpositions overcomes the wild and pathological behaviors of the inherent single variable functions, but retains the essence of Kolmogorov strategy of exact representation-an objective that Sprecher (Neural Netw. 144(2021)438-442) has actively pursued. We also discuss the implications of this new construction and demonstrate its applicability to related neural networks.