Video Killed the HD-Map: Predicting Multi-Agent Behavior Directly From Aerial Images
This addresses the problem of efficiently scaling traffic datasets for autonomous driving simulations, though it is incremental as it builds on existing multi-agent behavioral models.
The paper tackles the bottleneck of manually annotating HD maps for scaling up human traffic datasets in autonomous driving by proposing an aerial image-based map (AIM) representation that requires minimal annotation. The results show competitive multi-agent trajectory prediction performance, especially for pedestrians, compared to models using rasterized HD maps.
The development of algorithms that learn multi-agent behavioral models using human demonstrations has led to increasingly realistic simulations in the field of autonomous driving. In general, such models learn to jointly predict trajectories for all controlled agents by exploiting road context information such as drivable lanes obtained from manually annotated high-definition (HD) maps. Recent studies show that these models can greatly benefit from increasing the amount of human data available for training. However, the manual annotation of HD maps which is necessary for every new location puts a bottleneck on efficiently scaling up human traffic datasets. We propose an aerial image-based map (AIM) representation that requires minimal annotation and provides rich road context information for traffic agents like pedestrians and vehicles. We evaluate multi-agent trajectory prediction using the AIM by incorporating it into a differentiable driving simulator as an image-texture-based differentiable rendering module. Our results demonstrate competitive multi-agent trajectory prediction performance especially for pedestrians in the scene when using our AIM representation as compared to models trained with rasterized HD maps.