Fully Bayesian Analysis of the Relevance Vector Machine Classification for Imbalanced Data
This addresses the imbalanced data classification problem for machine learning practitioners, offering an incremental improvement over existing RVM methods.
The paper tackled the difficulty of Relevance Vector Machine (RVM) classification, which lacks a closed-form solution, by proposing a Fully Bayesian approach with hierarchical hyperprior structure, achieving high classification accuracy rates and outperforming other methods for imbalanced data.
Relevance Vector Machine (RVM) is a supervised learning algorithm extended from Support Vector Machine (SVM) based on the Bayesian sparsity model. Compared with the regression problem, RVM classification is difficult to be conducted because there is no closed-form solution for the weight parameter posterior. Original RVM classification algorithm used Newton's method in optimization to obtain the mode of weight parameter posterior then approximated it by a Gaussian distribution in Laplace's method. It would work but just applied the frequency methods in a Bayesian framework. This paper proposes a Generic Bayesian approach for the RVM classification. We conjecture that our algorithm achieves convergent estimates of the quantities of interest compared with the nonconvergent estimates of the original RVM classification algorithm. Furthermore, a Fully Bayesian approach with the hierarchical hyperprior structure for RVM classification is proposed, which improves the classification performance, especially in the imbalanced data problem. By the numeric studies, our proposed algorithms obtain high classification accuracy rates. The Fully Bayesian hierarchical hyperprior method outperforms the Generic one for the imbalanced data classification.