Predictive Approaches For Gaussian Process Classifier Model Selection
This addresses model selection for Gaussian process classifiers, particularly useful for imbalanced data, but is incremental as it builds on existing approximations and criteria.
The paper tackles the problem of model selection for Gaussian process classifiers by proposing a practical algorithm using Leave-One-Out predictive distributions with various optimization criteria, including NLP and F-measure, showing comparable or improved generalization performance on benchmark datasets, with F-measure significantly improving on several datasets.
In this paper we consider the problem of Gaussian process classifier (GPC) model selection with different Leave-One-Out (LOO) Cross Validation (CV) based optimization criteria and provide a practical algorithm using LOO predictive distributions with such criteria to select hyperparameters. Apart from the standard average negative logarithm of predictive probability (NLP), we also consider smoothed versions of criteria such as F-measure and Weighted Error Rate (WER), which are useful for handling imbalanced data. Unlike the regression case, LOO predictive distributions for the classifier case are intractable. We use approximate LOO predictive distributions arrived from Expectation Propagation (EP) approximation. We conduct experiments on several real world benchmark datasets. When the NLP criterion is used for optimizing the hyperparameters, the predictive approaches show better or comparable NLP generalization performance with existing GPC approaches. On the other hand, when the F-measure criterion is used, the F-measure generalization performance improves significantly on several datasets. Overall, the EP-based predictive algorithm comes out as an excellent choice for GP classifier model selection with different optimization criteria.