Uncertainty Estimation by Fisher Information-based Evidential Deep Learning
This work addresses a specific bottleneck in uncertainty estimation for deep learning, making it more reliable in practical applications, but it is incremental as it builds on existing evidential neural networks.
The paper tackles the problem of over-penalization in evidential deep learning for uncertainty estimation, particularly for mislabeled classes with high data uncertainty, by proposing a Fisher Information-based method that dynamically reweights loss terms and optimizes a PAC-Bayesian bound. The result is that the method consistently outperforms traditional EDL algorithms in multiple tasks, especially in few-shot classification settings.
Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. Recently proposed evidential neural networks explicitly account for different uncertainties by treating the network's outputs as evidence to parameterize the Dirichlet distribution, and achieve impressive performance in uncertainty estimation. However, for high data uncertainty samples but annotated with the one-hot label, the evidence-learning process for those mislabeled classes is over-penalized and remains hindered. To address this problem, we propose a novel method, Fisher Information-based Evidential Deep Learning ($\mathcal{I}$-EDL). In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes. The generalization ability of our network is further improved by optimizing the PAC-Bayesian bound. As demonstrated empirically, our proposed method consistently outperforms traditional EDL-related algorithms in multiple uncertainty estimation tasks, especially in the more challenging few-shot classification settings.