Three-Step Hierarchical Transformer for Multi-Pedestrian Trajectory Prediction
It addresses the need for scalable, flexible, and interpretable multi-pedestrian trajectory prediction in crowded environments.
The paper proposes a three-step hierarchical Transformer that separates temporal encoding, multimodal fusion, and social interaction reasoning for pedestrian trajectory prediction. It achieves state-of-the-art performance on JRDB and Urban datasets and competitive results on JTA.
Pedestrian trajectory prediction requires modeling temporal dynamics, multimodal cues, and social interactions in crowded environments. Existing methods often address these factors separately or entangle them in costly attention blocks, limiting scalability, flexibility, and interpretability. We propose a three-step hierarchical Transformer that explicitly separates temporal encoding, multimodal fusion, and scene-level interaction reasoning. Lightweight GRU summaries enable efficient cross-modal attention, while social attention over time--agent tokens captures inter-pedestrian influences at manageable cost. Experiments on JTA, JRDB, and the Pedestrians and Cyclists in Road Traffic dataset show state-of-the-art performance on real-world datasets (JRDB, Urban) and competitive results on JTA. Ablation and qualitative analyses confirm the contribution of each stage and the model's ability to anticipate complex behaviors such as early turning.