Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion
This addresses a safety-critical problem for autonomous driving systems in urban areas, though it appears incremental as it builds on existing datasets and prediction methods.
The paper tackles the problem of predicting pedestrians' sudden start/stop motions, which existing algorithms often fail at, by introducing a new benchmark called TRANS and a hybrid model that fuses multiple feature modalities. Their model sets a new benchmark on TRANS for pedestrian stop and go forecasting.
Forecasting pedestrians' future motions is essential for autonomous driving systems to safely navigate in urban areas. However, existing prediction algorithms often overly rely on past observed trajectories and tend to fail around abrupt dynamic changes, such as when pedestrians suddenly start or stop walking. We suggest that predicting these highly non-linear transitions should form a core component to improve the robustness of motion prediction algorithms. In this paper, we introduce the new task of pedestrian stop and go forecasting. Considering the lack of suitable existing datasets for it, we release TRANS, a benchmark for explicitly studying the stop and go behaviors of pedestrians in urban traffic. We build it from several existing datasets annotated with pedestrians' walking motions, in order to have various scenarios and behaviors. We also propose a novel hybrid model that leverages pedestrian-specific and scene features from several modalities, both video sequences and high-level attributes, and gradually fuses them to integrate multiple levels of context. We evaluate our model and several baselines on TRANS, and set a new benchmark for the community to work on pedestrian stop and go forecasting.