Determinantal Point Processes Implicitly Regularize Semi-parametric Regression Problems
This work provides a theoretical extension of kernel ridge regression results to semi-parametric models, which is incremental but useful for applications like geophysics and time series analysis.
The paper tackles the approximation of semi-parametric regression models using finite Determinantal Point Processes (DPPs), showing that determinantal sampling implicitly regularizes these problems and deriving a bound on the expected risk for the approximation.
Semi-parametric regression models are used in several applications which require comprehensibility without sacrificing accuracy. Typical examples are spline interpolation in geophysics, or non-linear time series problems, where the system includes a linear and non-linear component. We discuss here the use of a finite Determinantal Point Process (DPP) for approximating semi-parametric models. Recently, Barthelmé, Tremblay, Usevich, and Amblard introduced a novel representation of some finite DPPs. These authors formulated extended L-ensembles that can conveniently represent partial-projection DPPs and suggest their use for optimal interpolation. With the help of this formalism, we derive a key identity illustrating the implicit regularization effect of determinantal sampling for semi-parametric regression and interpolation. Also, a novel projected Nyström approximation is defined and used to derive a bound on the expected risk for the corresponding approximation of semi-parametric regression. This work naturally extends similar results obtained for kernel ridge regression.