k-Relevance Vectors: Considering Relevancy Beside Nearness
This work addresses classification accuracy improvements for machine learning practitioners, but it is incremental as it combines existing methods.
The study tackled improving k-nearest neighbor performance by combining it with relevance vector machines in kernel space, called k-relevance vectors, which prunes irrelevant attributes and introduces a new parameter for early stopping, resulting in highly competitive classification accuracy on UCI and computer vision datasets.
This study combines two different learning paradigms, k-nearest neighbor (k-NN) rule, as memory-based learning paradigm and relevance vector machines (RVM), as statistical learning paradigm. This combination is performed in kernel space and is called k-relevance vector (k-RV). The purpose is to improve the performance of k-NN rule. The proposed model significantly prunes irrelevant attributes. We also introduced a new parameter, responsible for early stopping of iterations in RVM. We show that the new parameter improves the classification accuracy of k-RV. Intensive experiments are conducted on several classification datasets from University of California Irvine (UCI) repository and two real datasets from computer vision domain. The performance of k-RV is highly competitive compared to a few state-of-the-arts in terms of classification accuracy.