Structured Basis Function Networks: Loss-Centric Multi-Hypothesis Ensembles with Controllable Diversity
This addresses the problem of structured ambiguity in uncertainty estimation for machine learning practitioners, offering a tunable mechanism, but it is incremental as it builds on existing multi-hypothesis and ensemble methods.
The paper tackles the lack of a unified framework for predictive uncertainty that combines multi-hypothesis diversity with principled ensemble aggregation, proposing Structured Basis Function Networks that link these through loss geometry, with experiments validating the approach across datasets.
Existing approaches to predictive uncertainty rely either on multi-hypothesis prediction, which promotes diversity but lacks principled aggregation, or on ensemble learning, which improves accuracy but rarely captures the structured ambiguity. This implicitly means that a unified framework consistent with the loss geometry remains absent. The Structured Basis Function Network addresses this gap by linking multi-hypothesis prediction and ensembling through centroidal aggregation induced by Bregman divergences. The formulation applies across regression and classification by aligning predictions with the geometry of the loss, and supports both a closed-form least-squares estimator and a gradient-based procedure for general objectives. A tunable diversity mechanism provides parametric control of the bias-variance-diversity trade-off, connecting multi-hypothesis generalisation with loss-aware ensemble aggregation. Experiments validate this relation and use the mechanism to study the complexity-capacity-diversity trade-off across datasets of increasing difficulty with deep-learning predictors.