An Additive Model View to Sparse Gaussian Process Classifier Design
This work addresses the challenge of efficient and effective SGPC design for large datasets, though it is incremental as it builds on existing methods like the information vector machine.
The paper tackles the problem of designing sparse Gaussian process classifiers (SGPCs) for better generalization by proposing a new basis vector selection method based on adaptive sampling, which improves performance at reduced computational cost, with experimental results showing better generalization on benchmark datasets, especially for smaller basis sets or difficult datasets.
We consider the problem of designing a sparse Gaussian process classifier (SGPC) that generalizes well. Viewing SGPC design as constructing an additive model like in boosting, we present an efficient and effective SGPC design method to perform a stage-wise optimization of a predictive loss function. We introduce new methods for two key components viz., site parameter estimation and basis vector selection in any SGPC design. The proposed adaptive sampling based basis vector selection method aids in achieving improved generalization performance at a reduced computational cost. This method can also be used in conjunction with any other site parameter estimation methods. It has similar computational and storage complexities as the well-known information vector machine and is suitable for large datasets. The hyperparameters can be determined by optimizing a predictive loss function. The experimental results show better generalization performance of the proposed basis vector selection method on several benchmark datasets, particularly for relatively smaller basis vector set sizes or on difficult datasets.