Bayesian Kernel and Mutual $k$-Nearest Neighbor Regression
This work offers incremental improvements for researchers in machine learning by adding Bayesian uncertainty estimation to existing nonparametric regression techniques.
The authors tackled the problem of nonparametric regression by proposing Bayesian extensions of kernel and mutual k-nearest neighbor methods, which provide distributions for estimates and hyperparameter selection. The results show that these methods asymptotically converge to the original ones and perform comparably or better on artificial and real-world datasets.
We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual $k$-nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is shown that both the proposed methods asymptotically converge to kernel and mutual $k$-nearest neighbor regression methods, respectively. The simulation results show that the proposed methods can select proper hyperparameters and are better than or comparable to the former methods for an artificial data set and a real world data set.