Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluation
This work addresses a specific technical bottleneck in uncertainty-aware neural networks for researchers and practitioners in machine learning, offering an incremental improvement to existing evidential models.
The paper tackled the problem of activation-dependent learning-freeze behavior in evidential deep learning models, which occurs when gradients become extremely small in low-evidence regions, and proposed a general family of activation functions and regularizers to mitigate this, achieving improved performance across multiple benchmarks including MNIST, CIFAR-10, CIFAR-100, Tiny-ImageNet, few-shot classification, and blind face restoration.
Evidential deep learning (EDL) models, based on Subjective Logic, introduce a principled and computationally efficient way to make deterministic neural networks uncertainty-aware. The resulting evidential models can quantify fine-grained uncertainty using learned evidence. However, the Subjective-Logic framework constrains evidence to be non-negative, requiring specific activation functions whose geometric properties can induce activation-dependent learning-freeze behavior: a regime where gradients become extremely small for samples mapped into low-evidence regions. We theoretically characterize this behavior and analyze how different evidential activations influence learning dynamics. Building on this analysis, we design a general family of activation functions and corresponding evidential regularizers that provide an alternative pathway for consistent evidence updates across activation regimes. Extensive experiments on four benchmark classification problems (MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet), two few-shot classification problems, and blind face restoration problem empirically validate the developed theory and demonstrate the effectiveness of the proposed generalized regularized evidential models.