When is there a Representer Theorem? Nondifferentiable Regularisers and Banach spaces
This work addresses a foundational problem in machine learning by generalizing the representer theorem, which is crucial for kernel methods, to broader mathematical settings, making it valuable for theoretical and computational advancements.
The paper extends necessary and sufficient conditions for the representer theorem from differentiable regularisers on Hilbert spaces to nondifferentiable regularisers on uniformly convex and smooth Banach spaces, providing a more complete answer to when such theorems exist.
We consider a general regularised interpolation problem for learning a parameter vector from data. The well known representer theorem says that under certain conditions on the regulariser there exists a solution in the linear span of the data points. This is the core of kernel methods in machine learning as it makes the problem computationally tractable. Necessary and sufficient conditions for differentiable regularisers on Hilbert spaces to admit a representer theorem have been proved. We extend those results to nondifferentiable regularisers on uniformly convex and uniformly smooth Banach spaces. This gives a (more) complete answer to the question when there is a representer theorem. We then note that for regularised interpolation in fact the solution is determined by the function space alone and independent of the regulariser, making the extension to Banach spaces even more valuable.