JointMotion: Joint Self-Supervision for Joint Motion Prediction
This work addresses motion prediction in autonomous driving, offering incremental improvements in accuracy and generalization for specific models and datasets.
The paper tackles joint motion prediction for self-driving vehicles by introducing JointMotion, a self-supervised pre-training method that jointly optimizes scene-level and instance-level objectives, resulting in reductions of joint final displacement error by 3% to 12% across different models and enabling transfer learning between datasets.
We present JointMotion, a self-supervised pre-training method for joint motion prediction in self-driving vehicles. Our method jointly optimizes a scene-level objective connecting motion and environments, and an instance-level objective to refine learned representations. Scene-level representations are learned via non-contrastive similarity learning of past motion sequences and environment context. At the instance level, we use masked autoencoding to refine multimodal polyline representations. We complement this with an adaptive pre-training decoder that enables JointMotion to generalize across different environment representations, fusion mechanisms, and dataset characteristics. Notably, our method reduces the joint final displacement error of Wayformer, HPTR, and Scene Transformer models by 3\%, 8\%, and 12\%, respectively; and enables transfer learning between the Waymo Open Motion and the Argoverse 2 Motion Forecasting datasets. Code: https://github.com/kit-mrt/future-motion