Hijacking Vision-and-Language Navigation Agents with Adversarial Environmental Attacks
This work addresses security vulnerabilities in assistive embodied agents, which could impact applications like manufacturing or in-home care, and is incremental as it builds on existing VLN settings with a novel attack method.
The authors tackled the problem of adversarial attacks on Vision-and-Language Navigation agents by developing a whitebox method that optimizes a 3D object's appearance to hijack agent behaviors, resulting in significant reductions in the agents' ability to follow instructions, such as inducing early termination or diversion along attacker-defined paths.
Assistive embodied agents that can be instructed in natural language to perform tasks in open-world environments have the potential to significantly impact labor tasks like manufacturing or in-home care -- benefiting the lives of those who come to depend on them. In this work, we consider how this benefit might be hijacked by local modifications in the appearance of the agent's operating environment. Specifically, we take the popular Vision-and-Language Navigation (VLN) task as a representative setting and develop a whitebox adversarial attack that optimizes a 3D attack object's appearance to induce desired behaviors in pretrained VLN agents that observe it in the environment. We demonstrate that the proposed attack can cause VLN agents to ignore their instructions and execute alternative actions after encountering the attack object -- even for instructions and agent paths not considered when optimizing the attack. For these novel settings, we find our attacks can induce early-termination behaviors or divert an agent along an attacker-defined multi-step trajectory. Under both conditions, environmental attacks significantly reduce agent capabilities to successfully follow user instructions.