Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approach
This provides robust uncertainty bounds for applications in machine learning and statistics where data has bounded noise without independence assumptions, but it is incremental as it builds on existing kernel and optimization methods.
The paper tackles the problem of establishing out-of-sample bounds for an unknown ground-truth function in reproducing kernel Hilbert spaces with bounded measurement noise, showing that computing tight, finite-sample uncertainty bounds reduces to solving parametric quadratically constrained linear programs.
The problem of establishing out-of-sample bounds for the values of an unkonwn ground-truth function is considered. Kernels and their associated Hilbert spaces are the main formalism employed herein along with an observational model where outputs are corrupted by bounded measurement noise. The noise can originate from any compactly supported distribution and no independence assumptions are made on the available data. In this setting, we show how computing tight, finite-sample uncertainty bounds amounts to solving parametric quadratically constrained linear programs. Next, properties of our approach are established and its relationship with another methods is studied. Numerical experiments are presented to exemplify how the theory can be applied in a number of scenarios, and to contrast it with other closed-form alternatives.