Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?
This work addresses scene understanding for navigation in robotics or AI agents, but it appears incremental as it builds on existing collision-based learning methods.
The paper tackles the problem of inferring scene geometry from collisions by introducing collision replay, which uses collision examples to supervise past observations, enabling convolutional neural networks to predict collision time distributions from images. The result shows that these distributions can be converted into distance functions for scene geometry, as demonstrated with an agent in a photorealistic simulator.
What does bumping into things in a scene tell you about scene geometry? In this paper, we investigate the idea of learning from collisions. At the heart of our approach is the idea of collision replay, where we use examples of a collision to provide supervision for observations at a past frame. We use collision replay to train convolutional neural networks to predict a distribution over collision time from new images. This distribution conveys information about the navigational affordances (e.g., corridors vs open spaces) and, as we show, can be converted into the distance function for the scene geometry. We analyze this approach with an agent that has noisy actuation in a photorealistic simulator.