A representer theorem for deep kernel learning
This work provides a mathematical foundation for deep learning methods, addressing a theoretical gap for researchers in machine learning, though it is incremental as it builds on existing representer theorem concepts.
The paper tackles the problem of analyzing machine learning algorithms based on compositions of functions by providing finite-sample and infinite-sample representer theorems for deep kernel learning, enabling the recasting of infinite-dimensional minimization problems into finite-dimensional ones that can be solved with nonlinear optimization algorithms.
In this paper we provide a finite-sample and an infinite-sample representer theorem for the concatenation of (linear combinations of) kernel functions of reproducing kernel Hilbert spaces. These results serve as mathematical foundation for the analysis of machine learning algorithms based on compositions of functions. As a direct consequence in the finite-sample case, the corresponding infinite-dimensional minimization problems can be recast into (nonlinear) finite-dimensional minimization problems, which can be tackled with nonlinear optimization algorithms. Moreover, we show how concatenated machine learning problems can be reformulated as neural networks and how our representer theorem applies to a broad class of state-of-the-art deep learning methods.