Zero Memory Overhead Approach for Protecting Vision Transformer Parameters
This addresses reliability issues in ViTs for safety-critical domains, but it is incremental as it adapts existing fault tolerance concepts to a specific model type.
The paper tackles the problem of bit-flip faults in Vision Transformer (ViT) parameters stored in memory for safety-critical applications like autonomous driving, by introducing a fault tolerance technique that uses parity bits in least significant bits for error detection and zeroing out affected parameters, achieving up to three orders of magnitude improvement in robustness with zero memory overhead.
Vision Transformers (ViTs) have demonstrated superior performance over Convolutional Neural Networks (CNNs) in various vision-related tasks such as classification, object detection, and segmentation due to their use of self-attention mechanisms. As ViTs become more popular in safety-critical applications like autonomous driving, ensuring their correct functionality becomes essential, especially in the presence of bit-flip faults in their parameters stored in memory. In this paper, a fault tolerance technique is introduced to protect ViT parameters against bit-flip faults with zero memory overhead. Since the least significant bits of parameters are not critical for model accuracy, replacing the LSB with a parity bit provides an error detection mechanism without imposing any overhead on the model. When faults are detected, affected parameters are masked by zeroing out, as most parameters in ViT models are near zero, effectively preventing accuracy degradation. This approach enhances reliability across ViT models, improving the robustness of parameters to bit-flips by up to three orders of magnitude, making it an effective zero-overhead solution for fault tolerance in critical applications.