Enhancing Reliability of Neural Networks at the Edge: Inverted Normalization with Stochastic Affine Transformations
This work addresses reliability issues for safety-critical edge applications using memristor-based computing, though it is incremental as it builds on existing Bayesian Neural Networks and normalization techniques.
The paper tackles the problem of enhancing robustness and inference accuracy of Bayesian Neural Networks in memristor-based in-memory computing architectures, which are prone to non-idealities like noise and variations, by introducing a novel normalization layer with stochastic affine transformations, resulting in up to 58.11% improvement in graceful degradation of inference accuracy.
Bayesian Neural Networks (BayNNs) naturally provide uncertainty in their predictions, making them a suitable choice in safety-critical applications. Additionally, their realization using memristor-based in-memory computing (IMC) architectures enables them for resource-constrained edge applications. In addition to predictive uncertainty, however, the ability to be inherently robust to noise in computation is also essential to ensure functional safety. In particular, memristor-based IMCs are susceptible to various sources of non-idealities such as manufacturing and runtime variations, drift, and failure, which can significantly reduce inference accuracy. In this paper, we propose a method to inherently enhance the robustness and inference accuracy of BayNNs deployed in IMC architectures. To achieve this, we introduce a novel normalization layer combined with stochastic affine transformations. Empirical results in various benchmark datasets show a graceful degradation in inference accuracy, with an improvement of up to $58.11\%$.