Evaluating Prediction Uncertainty Estimates from BatchEnsemble
This addresses uncertainty estimation for deep learning practitioners, offering a more efficient alternative to existing methods, though it is incremental as it builds on ensemble techniques.
The paper tackled the problem of uncertainty estimation in deep learning by evaluating BatchEnsemble as a scalable method, showing it matches deep ensembles' performance and outperforms Monte Carlo dropout, with GRUBE achieving similar or better results using fewer parameters and reduced computational time.
Deep learning models struggle with uncertainty estimation. Many approaches are either computationally infeasible or underestimate uncertainty. We investigate \textit{BatchEnsemble} as a general and scalable method for uncertainty estimation across both tabular and time series tasks. To extend BatchEnsemble to sequential modeling, we introduce GRUBE, a novel BatchEnsemble GRU cell. We compare the BatchEnsemble to Monte Carlo dropout and deep ensemble models. Our results show that BatchEnsemble matches the uncertainty estimation performance of deep ensembles, and clearly outperforms Monte Carlo dropout. GRUBE achieves similar or better performance in both prediction and uncertainty estimation. These findings show that BatchEnsemble and GRUBE achieve similar performance with fewer parameters and reduced training and inference time compared to traditional ensembles.