Comparison of Data Reduction Criteria for Online Gaussian Processes
This work provides incremental guidelines for practitioners using online Gaussian Processes in dynamic system identification and similar applications.
The paper tackled the computational challenge of Gaussian Processes in streaming scenarios by comparing several data reduction criteria, finding that the proposed acceptance criteria improve filtering of redundant datapoints on benchmark functions and real-world datasets.
Gaussian Processes (GPs) are widely used for regression and system identification due to their flexibility and ability to quantify uncertainty. However, their computational complexity limits their applicability to small datasets. Moreover in a streaming scenario, more and more datapoints accumulate which is intractable even for Sparse GPs. Online GPs aim to alleviate this problem by e.g. defining a maximum budget of datapoints and removing redundant datapoints. This work provides a unified comparison of several reduction criteria, analyzing both their computational complexity and reduction behavior. The criteria are evaluated on benchmark functions and real-world datasets, including dynamic system identification tasks. Additionally, acceptance criteria are proposed to further filter out redundant datapoints. This work yields practical guidelines for choosing a suitable criterion for an online GP algorithm.