PathFinder: Attention-Driven Dynamic Non-Line-of-Sight Tracking with a Mobile Robot
This enables low-cost NLOS tracking for applications like rescue operations and autonomous vehicles, but it is incremental as it builds on existing NLOS imaging research by adapting it to dynamic environments.
The paper tackles the problem of non-line-of-sight (NLOS) imaging with a moving camera, proposing PathFinder, a data-driven method using a standard RGB camera on a mobile robot to estimate the 2D trajectory of a hidden person in real-time, validated with drone-captured in-the-wild scenes.
The study of non-line-of-sight (NLOS) imaging is growing due to its many potential applications, including rescue operations and pedestrian detection by self-driving cars. However, implementing NLOS imaging on a moving camera remains an open area of research. Existing NLOS imaging methods rely on time-resolved detectors and laser configurations that require precise optical alignment, making it difficult to deploy them in dynamic environments. This work proposes a data-driven approach to NLOS imaging, PathFinder, that can be used with a standard RGB camera mounted on a small, power-constrained mobile robot, such as an aerial drone. Our experimental pipeline is designed to accurately estimate the 2D trajectory of a person who moves in a Manhattan-world environment while remaining hidden from the camera's field-of-view. We introduce a novel approach to process a sequence of dynamic successive frames in a line-of-sight (LOS) video using an attention-based neural network that performs inference in real-time. The method also includes a preprocessing selection metric that analyzes images from a moving camera which contain multiple vertical planar surfaces, such as walls and building facades, and extracts planes that return maximum NLOS information. We validate the approach on in-the-wild scenes using a drone for video capture, thus demonstrating low-cost NLOS imaging in dynamic capture environments.