Modeling 3D Pedestrian-Vehicle Interactions for Vehicle-Conditioned Pose Forecasting
This work addresses the challenge of safe autonomous driving in urban environments by enhancing pedestrian motion prediction with vehicle interactions, though it is incremental as it builds on existing datasets and architectures.
The paper tackles the problem of predicting pedestrian motion in autonomous driving by introducing a 3D vehicle-conditioned pedestrian pose forecasting framework that incorporates surrounding vehicle information, resulting in substantial improvements in forecasting accuracy as demonstrated through extensive experiments.
Accurately predicting pedestrian motion is crucial for safe and reliable autonomous driving in complex urban environments. In this work, we present a 3D vehicle-conditioned pedestrian pose forecasting framework that explicitly incorporates surrounding vehicle information. To support this, we enhance the Waymo-3DSkelMo dataset with aligned 3D vehicle bounding boxes, enabling realistic modeling of multi-agent pedestrian-vehicle interactions. We introduce a sampling scheme to categorize scenes by pedestrian and vehicle count, facilitating training across varying interaction complexities. Our proposed network adapts the TBIFormer architecture with a dedicated vehicle encoder and pedestrian-vehicle interaction cross-attention module to fuse pedestrian and vehicle features, allowing predictions to be conditioned on both historical pedestrian motion and surrounding vehicles. Extensive experiments demonstrate substantial improvements in forecasting accuracy and validate different approaches for modeling pedestrian-vehicle interactions, highlighting the importance of vehicle-aware 3D pose prediction for autonomous driving. Code is available at: https://github.com/GuangxunZhu/VehCondPose3D