Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
This enables accurate, real-time tracking for VR/AR and video games in challenging high-speed or high dynamic range conditions.
The paper tackles the problem of low-latency 6-DOF camera tracking for VR/AR applications by using event cameras with photometric depth maps, achieving tracking upon each event arrival to virtually eliminate latency and enabling operation in high-speed motion scenes inaccessible to standard cameras.
Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high-speed motions or in scenes characterized by high dynamic range. These features, along with a very low power consumption, make event cameras an ideal complement to standard cameras for VR/AR and video game applications. With these applications in mind, this paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map (i.e., intensity plus depth information) built via classic dense reconstruction pipelines. Our approach tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency. We successfully evaluate the method in both indoor and outdoor scenes and show that---because of the technological advantages of the event camera---our pipeline works in scenes characterized by high-speed motion, which are still unaccessible to standard cameras.