Scalable and Efficient Methods for Uncertainty Estimation and Reduction in Deep Learning
It addresses reliability issues for deploying neural networks in resource-constrained safety-critical systems, such as automated decision-making, but appears incremental as it builds on existing techniques like dropout and variational inference.
The paper tackles uncertainty in neural networks for safety-critical systems by developing scalable methods for uncertainty estimation and reduction, focusing on Computation-in-Memory with resistive memories, resulting in enhanced out-of-distribution detection, inference accuracy, and energy efficiency.
Neural networks (NNs) can achieved high performance in various fields such as computer vision, and natural language processing. However, deploying NNs in resource-constrained safety-critical systems has challenges due to uncertainty in the prediction caused by out-of-distribution data, and hardware non-idealities. To address the challenges of deploying NNs in resource-constrained safety-critical systems, this paper summarizes the (4th year) PhD thesis work that explores scalable and efficient methods for uncertainty estimation and reduction in deep learning, with a focus on Computation-in-Memory (CIM) using emerging resistive non-volatile memories. We tackle the inherent uncertainties arising from out-of-distribution inputs and hardware non-idealities, crucial in maintaining functional safety in automated decision-making systems. Our approach encompasses problem-aware training algorithms, novel NN topologies, and hardware co-design solutions, including dropout-based \emph{binary} Bayesian Neural Networks leveraging spintronic devices and variational inference techniques. These innovations significantly enhance OOD data detection, inference accuracy, and energy efficiency, thereby contributing to the reliability and robustness of NN implementations.