Scalable Multi-Class Bayesian Support Vector Machines for Structured and Unstructured Data
This work provides a scalable Bayesian method for classification with uncertainty estimation, addressing needs in domains like active learning and adversarial detection, though it is incremental as it builds on existing SVM and Bayesian frameworks.
The authors tackled the problem of multi-class classification for both structured and unstructured data by introducing a Bayesian multi-class support vector machine with variational inference and inducing point approximations, achieving improved training time and accuracy on 68 structured and two unstructured datasets compared to competitors.
We introduce a new Bayesian multi-class support vector machine by formulating a pseudo-likelihood for a multi-class hinge loss in the form of a location-scale mixture of Gaussians. We derive a variational-inference-based training objective for gradient-based learning. Additionally, we employ an inducing point approximation which scales inference to large data sets. Furthermore, we develop hybrid Bayesian neural networks that combine standard deep learning components with the proposed model to enable learning for unstructured data. We provide empirical evidence that our model outperforms the competitor methods with respect to both training time and accuracy in classification experiments on 68 structured and two unstructured data sets. Finally, we highlight the key capability of our model in yielding prediction uncertainty for classification by demonstrating its effectiveness in the tasks of large-scale active learning and detection of adversarial images.