Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting
This work addresses generalizability and uncertainty issues in trajectory prediction for autonomous vehicles, representing a novel method for a known bottleneck rather than an incremental improvement.
The paper tackles the problem of vehicle trajectory prediction by proposing PerReg+, a framework that introduces dual-level representation learning and adaptive prompting to improve generalizability and handle complex interactions, achieving state-of-the-art performance on multiple datasets with a 6.8% error reduction on smaller datasets and an 11.8% reduction in B-FDE in cross-domain tests.
Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions. It is often due to limitations like complex architectures customized for a specific dataset and inefficient multimodal handling. We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details. Additionally, our approach of reconstructing segmentlevel trajectories and lane segments from masked inputs with query drop, enables effective use of contextual information and improves generalization; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation. PerReg+ sets a new state-of-the-art performance on nuScenes [1], Argoverse 2 [2], and Waymo Open Motion Dataset (WOMD) [3]. Remarkable, our pretrained model reduces the error by 6.8% on smaller datasets, and multi-dataset training enhances generalization. In cross-domain tests, PerReg+ reduces B-FDE by 11.8% compared to its non-pretrained variant.