Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces
This work addresses the need for sparse learning methods in machine learning by providing theoretical foundations for sparsity in RKBSs, though it appears incremental as it builds on existing RKBS frameworks.
The paper tackles the problem of identifying which reproducing kernel Banach spaces (RKBSs) promote sparsity in learning solutions, establishing explicit representer theorems for minimum norm interpolation and regularization problems, and showing that specific RKBSs like ℓ₁(ℕ) and measure spaces yield sparse kernel representations with fewer terms than observed data.
Sparsity of a learning solution is a desirable feature in machine learning. Certain reproducing kernel Banach spaces (RKBSs) are appropriate hypothesis spaces for sparse learning methods. The goal of this paper is to understand what kind of RKBSs can promote sparsity for learning solutions. We consider two typical learning models in an RKBS: the minimum norm interpolation (MNI) problem and the regularization problem. We first establish an explicit representer theorem for solutions of these problems, which represents the extreme points of the solution set by a linear combination of the extreme points of the subdifferential set, of the norm function, which is data-dependent. We then propose sufficient conditions on the RKBS that can transform the explicit representation of the solutions to a sparse kernel representation having fewer terms than the number of the observed data. Under the proposed sufficient conditions, we investigate the role of the regularization parameter on sparsity of the regularized solutions. We further show that two specific RKBSs: the sequence space $\ell_1(\mathbb{N})$ and the measure space can have sparse representer theorems for both MNI and regularization models.