Hidden Footprints: Learning Contextual Walkability from 3D Human Trails
This work solves the problem of learning contextual walkability for applications like autonomous driving and human behavior analysis, but it is incremental as it builds on existing datasets and methods.
The paper tackles the problem of predicting walkable regions in scenes from single images, addressing challenges of semantic ambiguity and sparse labeled data by augmenting existing datasets with 3D-propagated hidden footprints and using a training strategy with class-balanced and adversarial losses. The model achieves superior performance on Waymo and Cityscapes datasets compared to baselines and state-of-the-art models.
Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis. Yet learning a computational model for this purpose is challenging due to semantic ambiguity and a lack of labeled data: current datasets only tell you where people are, not where they could be. We tackle this problem by leveraging information from existing datasets, without additional labeling. We first augment the set of valid, labeled walkable regions by propagating person observations between images, utilizing 3D information to create what we call hidden footprints. However, this augmented data is still sparse. We devise a training strategy designed for such sparse labels, combining a class-balanced classification loss with a contextual adversarial loss. Using this strategy, we demonstrate a model that learns to predict a walkability map from a single image. We evaluate our model on the Waymo and Cityscapes datasets, demonstrating superior performance compared to baselines and state-of-the-art models.