Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics
For robotics researchers deploying VLA models, this work highlights a practical vulnerability where adversarial patches can cause long-horizon failures under realistic partial observability.
The paper introduces a partially observable adversarial patch attack on VLA models in robotics, where the adversary uses only a short trajectory prefix to generate a fixed patch that disrupts perception and control, significantly reducing task success rates in both simulation and real-world settings.
Vision-language-action (VLA) models are gaining attention in robotics, yet their robustness to adversarial attacks remains largely unexplored. Existing work shows that adversarial patches can mislead VLA-based robots but assumes full access to the entire execution trajectory, an unrealistic requirement in practice. We address this limitation by formulating a partially observable threat model, where the adversary can exploit only a short prefix of the trajectory to generate a fixed patch applied to all subsequent frames. Under this setting, we propose a two-phase framework. First, we localize the patch using the model's attention maps to identify visually critical regions that correspond to the full instruction. Then, we optimize the patch to disrupt the semantic grounding of target objects and increase the curvature of action trajectories, thereby compounding failures in both perception and control. Extensive experiments in simulation and real-world robotic environments show that our method sustains adversarial effects under partial observability, inducing long-horizon disruptions and significantly reducing task success rates.