Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks
This provides a practical solution for improving risk-awareness in deep neural networks, particularly in complex perception tasks, though it is incremental as it builds on existing uncertainty estimation methods.
The authors tackled the problem of sudden and catastrophic failures in deep neural networks by introducing Capsa, a unified framework for quantifying multiple forms of risk, which enables easy composition of aleatoric uncertainty, epistemic uncertainty, and bias estimation in a single procedure.
The modern pervasiveness of large-scale deep neural networks (NNs) is driven by their extraordinary performance on complex problems but is also plagued by their sudden, unexpected, and often catastrophic failures, particularly on challenging scenarios. Existing algorithms that provide risk-awareness to NNs are complex and ad-hoc. Specifically, these methods require significant engineering changes, are often developed only for particular settings, and are not easily composable. Here we present capsa, a framework for extending models with risk-awareness. Capsa provides a methodology for quantifying multiple forms of risk and composing different algorithms together to quantify different risk metrics in parallel. We validate capsa by implementing state-of-the-art uncertainty estimation algorithms within the capsa framework and benchmarking them on complex perception datasets. We demonstrate capsa's ability to easily compose aleatoric uncertainty, epistemic uncertainty, and bias estimation together in a single procedure, and show how this approach provides a comprehensive awareness of NN risk.