Incremental Sparse Bayesian Ordinal Regression
This work addresses the time-consuming bottleneck in ordinal regression for multi-label learning, offering a more scalable solution for large-scale datasets, though it is incremental as it builds on existing basis function methods.
The paper tackles the computational inefficiency of basis function-based ordinal regression methods by proposing an incremental sparse Bayesian approach (ISBOR) that sequentially learns relevant basis functions and avoids large matrix inverses, demonstrating improved efficiency and effectiveness on synthetic and real-world datasets.
Ordinal Regression (OR) aims to model the ordering information between different data categories, which is a crucial topic in multi-label learning. An important class of approaches to OR models the problem as a linear combination of basis functions that map features to a high dimensional non-linear space. However, most of the basis function-based algorithms are time consuming. We propose an incremental sparse Bayesian approach to OR tasks and introduce an algorithm to sequentially learn the relevant basis functions in the ordinal scenario. Our method, called Incremental Sparse Bayesian Ordinal Regression (ISBOR), automatically optimizes the hyper-parameters via the type-II maximum likelihood method. By exploiting fast marginal likelihood optimization, ISBOR can avoid big matrix inverses, which is the main bottleneck in applying basis function-based algorithms to OR tasks on large-scale datasets. We show that ISBOR can make accurate predictions with parsimonious basis functions while offering automatic estimates of the prediction uncertainty. Extensive experiments on synthetic and real word datasets demonstrate the efficiency and effectiveness of ISBOR compared to other basis function-based OR approaches.