Optimal Binary Classifier Aggregation for General Losses
This provides a theoretically optimal solution for ensemble aggregation in semi-supervised learning, though it appears incremental as it builds on existing analysis for misclassification error.
The paper tackles the problem of aggregating an ensemble of predictors to minimize prediction loss in semi-supervised binary classification, extending prior work to a general class of loss functions and achieving minimax optimality without relaxations.
We address the problem of aggregating an ensemble of predictors with known loss bounds in a semi-supervised binary classification setting, to minimize prediction loss incurred on the unlabeled data. We find the minimax optimal predictions for a very general class of loss functions including all convex and many non-convex losses, extending a recent analysis of the problem for misclassification error. The result is a family of semi-supervised ensemble aggregation algorithms which are as efficient as linear learning by convex optimization, but are minimax optimal without any relaxations. Their decision rules take a form familiar in decision theory -- applying sigmoid functions to a notion of ensemble margin -- without the assumptions typically made in margin-based learning.