Divergent Ensemble Networks: Enhancing Uncertainty Estimation with Shared Representations and Independent Branching
This is an incremental improvement for machine learning practitioners seeking more efficient uncertainty estimation in neural networks.
The paper tackles the problem of parameter redundancy and computational inefficiency in conventional ensemble methods by proposing the Divergent Ensemble Network (DEN), which combines shared representation learning with independent branching to reduce parameters while maintaining ensemble diversity.
Ensemble learning has proven effective in improving predictive performance and estimating uncertainty in neural networks. However, conventional ensemble methods often suffer from redundant parameter usage and computational inefficiencies due to entirely independent network training. To address these challenges, we propose the Divergent Ensemble Network (DEN), a novel architecture that combines shared representation learning with independent branching. DEN employs a shared input layer to capture common features across all branches, followed by divergent, independently trainable layers that form an ensemble. This shared-to-branching structure reduces parameter redundancy while maintaining ensemble diversity, enabling efficient and scalable learning.