Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions
This work addresses function approximation on spheres for researchers in machine learning and approximation theory, but it is incremental as it extends known methods to a specific activation function class.
The paper tackles the problem of approximating functions on spheres using zonal function networks with non-positive definite activation functions like |t|, establishing approximation properties for a defined smoothness class with centers independent of the target function and coefficients derived from training data.
A zonal function (ZF) network on the $q$ dimensional sphere $\mathbb{S}^q$ is a network of the form $\mathbf{x}\mapsto \sum_{k=1}^n a_kφ(\mathbf{x}\cdot\mathbf{x}_k)$ where $φ:[-1,1]\to\mathbf{R}$ is the activation function, $\mathbf{x}_k\in\mathbb{S}^q$ are the centers, and $a_k\in\mathbb{R}$. While the approximation properties of such networks are well studied in the context of positive definite activation functions, recent interest in deep and shallow networks motivate the study of activation functions of the form $φ(t)=|t|$, which are not positive definite. In this paper, we define an appropriate smoothess class and establish approximation properties of such networks for functions in this class. The centers can be chosen independently of the target function, and the coefficients are linear combinations of the training data. The constructions preserve rotational symmetries.