Felix Schwenninger

Thomas Ransford, Felix Schwenninger

Crouzeix and Palencia recently showed that the numerical range of a Hilbert-space operator is a $(1+\sqrt2)$-spectral set for the operator. One of the principal ingredients of their proof can be formulated as an abstract functional-analysis lemma. We give a new short proof of the lemma and show that, in the context of this lemma, the constant $(1+\sqrt2)$ is sharp.

Len Spek, Tjeerd Jan Heeringa, Felix Schwenninger et al.

Reproducing Kernel Hilbert spaces (RKHS) have been a very successful tool in various areas of machine learning. Recently, Barron spaces have been used to prove bounds on the generalisation error for neural networks. Unfortunately, Barron spaces cannot be understood in terms of RKHS due to the strong nonlinear coupling of the weights. This can be solved by using the more general Reproducing Kernel Banach spaces (RKBS). We show that these Barron spaces belong to a class of integral RKBS. This class can also be understood as an infinite union of RKHS spaces. Furthermore, we show that the dual space of such RKBSs, is again an RKBS where the roles of the data and parameters are interchanged, forming an adjoint pair of RKBSs including a reproducing kernel. This allows us to construct the saddle point problem for neural networks, which can be used in the whole field of primal-dual optimisation.

Tjeerd Jan Heeringa, Len Spek, Felix Schwenninger et al.

The approximation properties of infinitely wide shallow neural networks heavily depend on the choice of the activation function. To understand this influence, we study embeddings between Barron spaces with different activation functions. These embeddings are proven by providing push-forward maps on the measures $μ$ used to represent functions $f$. An activation function of particular interest is the rectified power unit ($\operatorname{RePU}$) given by $\operatorname{RePU}_s(x)=\max(0,x)^s$. For many commonly used activation functions, the well-known Taylor remainder theorem can be used to construct a push-forward map, which allows us to prove the embedding of the associated Barron space into a Barron space with a $\operatorname{RePU}$ as activation function. Moreover, the Barron spaces associated with the $\operatorname{RePU}_s$ have a hierarchical structure similar to the Sobolev spaces $H^m$.

Felix Schwenninger

We show $H^{\infty}$-functional calculus estimates for Tadmor-Ritt operators (also known as Ritt operators), which generalize and improve results by Vitse. These estimates are in conformity with the best known power-bounds for Tadmor-Ritt operators in terms of the constant dependence. Furthermore, it is shown how discrete square function estimates influence the estimates.