Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty
This addresses the challenge of accurate trajectory prediction in autonomous driving by leveraging perceptual uncertainty, though it is incremental as it builds on existing forecasting methods.
The paper tackles the problem of trajectory forecasting for safe robot navigation by incorporating agents' class uncertainty, which is typically neglected, and shows that this significantly improves performance in uncertain scenarios, enabling new capabilities like counterfactual predictions.
Reasoning about the future behavior of other agents is critical to safe robot navigation. The multiplicity of plausible futures is further amplified by the uncertainty inherent to agent state estimation from data, including positions, velocities, and semantic class. Forecasting methods, however, typically neglect class uncertainty, conditioning instead only on the agent's most likely class, even though perception models often return full class distributions. To exploit this information, we present HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities. We additionally present PUP, a new challenging real-world autonomous driving dataset, to investigate the impact of Perceptual Uncertainty in Prediction. It contains challenging crowded scenes with unfiltered agent class probabilities that reflect the long-tail of current state-of-the-art perception systems. We demonstrate that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty, and enables new forecasting capabilities such as counterfactual predictions.