Towards Viewpoint-Robust End-to-End Autonomous Driving with 3D Foundation Model Priors
This addresses viewpoint robustness for scalable end-to-end autonomous driving, but it is incremental as gains are limited under some perturbations.
The paper tackled the problem of robust trajectory planning for autonomous driving under camera viewpoint changes by leveraging geometric priors from a 3D foundation model, resulting in reduced performance degradation under most perturbation conditions, with clear improvements under pitch and height perturbations.
Robust trajectory planning under camera viewpoint changes is important for scalable end-to-end autonomous driving. However, existing models often depend heavily on the camera viewpoints seen during training. We investigate an augmentation-free approach that leverages geometric priors from a 3D foundation model. The method injects per-pixel 3D positions derived from depth estimates as positional embeddings and fuses intermediate geometric features through cross-attention. Experiments on the VR-Drive camera viewpoint perturbation benchmark show reduced performance degradation under most perturbation conditions, with clear improvements under pitch and height perturbations. Gains under longitudinal translation are smaller, suggesting that more viewpoint-agnostic integration is needed for robustness to camera viewpoint changes.