FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions
This work addresses error resilience for DNNs in safety-critical applications like healthcare and autonomous vehicles, offering a low-cost solution that is incremental over existing activation-based techniques.
The paper tackles the problem of error resilience in deep neural networks for safety-critical edge devices by proposing FitAct, which uses fine-grained post-trainable activation functions to bound neuron values and prevent fault propagation. Experimental results show that FitAct outperforms state-of-the-art methods like Clip-Act and Ranger across various fault rates with manageable overheads.
Deep neural networks (DNNs) are increasingly being deployed in safety-critical systems such as personal healthcare devices and self-driving cars. In such DNN-based systems, error resilience is a top priority since faults in DNN inference could lead to mispredictions and safety hazards. For latency-critical DNN inference on resource-constrained edge devices, it is nontrivial to apply conventional redundancy-based fault tolerance techniques. In this paper, we propose FitAct, a low-cost approach to enhance the error resilience of DNNs by deploying fine-grained post-trainable activation functions. The main idea is to precisely bound the activation value of each individual neuron via neuron-wise bounded activation functions so that it could prevent fault propagation in the network. To avoid complex DNN model re-training, we propose to decouple the accuracy training and resilience training and develop a lightweight post-training phase to learn these activation functions with precise bound values. Experimental results on widely used DNN models such as AlexNet, VGG16, and ResNet50 demonstrate that FitAct outperforms state-of-the-art studies such as Clip-Act and Ranger in enhancing the DNN error resilience for a wide range of fault rates while adding manageable runtime and memory space overheads.