Multi-Kernel Regression with Sparsity Constraint
This work provides a theoretical framework for sparsity-constrained regression, which is incremental as it builds on existing regularization methods.
The paper tackles the problem of supervised learning with generalized total-variation regularization by formulating it in a Banach space and identifying admissible kernel functions, resulting in a multi-kernel expansion with adaptive positions where the number of active kernels is bounded by data points and coefficients are penalized via an ℓ₁ norm.
In this paper, we provide a Banach-space formulation of supervised learning with generalized total-variation (gTV) regularization. We identify the class of kernel functions that are admissible in this framework. Then, we propose a variation of supervised learning in a continuous-domain hybrid search space with gTV regularization. We show that the solution admits a multi-kernel expansion with adaptive positions. In this representation, the number of active kernels is upper-bounded by the number of data points while the gTV regularization imposes an $\ell_1$ penalty on the kernel coefficients. Finally, we illustrate numerically the outcome of our theory.