On the Relationship between Online Gaussian Process Regression and Kernel Least Mean Squares Algorithms
This work provides insights into the probabilistic underpinnings of online learning algorithms for researchers in machine learning, though it is incremental as it clarifies existing relationships rather than introducing new methods.
The paper investigates the connection between online Gaussian process regression and kernel least mean squares algorithms, revealing that KLMS algorithms operate under a fixed posterior covariance assumption with specific parametric models, which explains their uncertainty handling and performance variations.
We study the relationship between online Gaussian process (GP) regression and kernel least mean squares (KLMS) algorithms. While the latter have no capacity of storing the entire posterior distribution during online learning, we discover that their operation corresponds to the assumption of a fixed posterior covariance that follows a simple parametric model. Interestingly, several well-known KLMS algorithms correspond to specific cases of this model. The probabilistic perspective allows us to understand how each of them handles uncertainty, which could explain some of their performance differences.