When is there a Representer Theorem? Reflexive Banach spaces
This work addresses a foundational theoretical gap in machine learning by extending representer theorems from Hilbert to Banach spaces, which is incremental but clarifies the minimal conditions needed for computational tractability in kernel methods.
The paper tackles the problem of determining when a representer theorem exists for regularised interpolation in learning, proving necessary and sufficient conditions in reflexive Banach spaces and showing that reflexivity is the minimal requirement, with the solution being independent of the regulariser in such cases.
We consider a general regularised interpolation problem for learning a parameter vector from data. The well known representer theorem says that under certain conditions on the regulariser there exists a solution in the linear span of the data points. This is at the core of kernel methods in machine learning as it makes the problem computationally tractable. Most literature deals only with sufficient conditions for representer theorems in Hilbert spaces. We prove necessary and sufficient conditions for the existence of representer theorems in reflexive Banach spaces and illustrate why in a sense reflexivity is the minimal requirement on the function space. We further show that if the learning relies on the linear representer theorem, then the solution is independent of the regulariser and in fact determined by the function space alone. This in particular shows the value of generalising Hilbert space learning theory to Banach spaces.