SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectors
This work addresses safety risks in 3D perception systems for autonomous driving by preemptively discovering novel challenging objects, though it is incremental as it builds on existing gradient-based methods for adversarial generation.
The authors tackled the problem of identifying vulnerabilities in 3D object detectors by developing SHIFT3D, a differentiable pipeline that generates structurally plausible 3D shapes to trick these detectors, resulting in interpretable failure modes for safety-critical applications like autonomous driving.
We present SHIFT3D, a differentiable pipeline for generating 3D shapes that are structurally plausible yet challenging to 3D object detectors. In safety-critical applications like autonomous driving, discovering such novel challenging objects can offer insight into unknown vulnerabilities of 3D detectors. By representing objects with a signed distanced function (SDF), we show that gradient error signals allow us to smoothly deform the shape or pose of a 3D object in order to confuse a downstream 3D detector. Importantly, the objects generated by SHIFT3D physically differ from the baseline object yet retain a semantically recognizable shape. Our approach provides interpretable failure modes for modern 3D object detectors, and can aid in preemptive discovery of potential safety risks within 3D perception systems before these risks become critical failures.