LoRA as a Flexible Framework for Securing Large Vision Systems
This addresses security vulnerabilities in autonomous driving systems, offering a flexible and efficient solution for patching large pre-trained models as new threats emerge, though it is incremental in applying existing parameter-efficient fine-tuning methods.
The paper tackles the problem of adversarial attacks on autonomous driving vision systems by proposing a lightweight security patch using low-rank adaptation (LoRA), which improves classification accuracy by up to 78.01% against adversarial examples.
Adversarial attacks have emerged as a critical threat to autonomous driving systems. These attacks exploit the underlying neural network, allowing small -- nearly invisible -- perturbations to completely alter the behavior of such systems in potentially malicious ways. E.g., causing a traffic sign classification network to misclassify a stop sign as a speed limit sign. Prior working in hardening such systems to adversarial attacks have looked at robust training of the system or adding additional pre-processing steps to the input pipeline. Such solutions either have a hard time generalizing, require knowledge of the adversarial attacks during training, or are computationally undesirable. Instead, we propose to take insights for parameter efficient fine-tuning and use low-rank adaptation (LoRA) to train a lightweight security patch -- enabling us to dynamically patch a large preexisting vision system as new vulnerabilities are discovered. We demonstrate that our framework can patch a pre-trained model to improve classification accuracy by up to 78.01% in the presence of adversarial examples.