RADU: Ray-Aligned Depth Update Convolutions for ToF Data Denoising
This addresses noise reduction in ToF cameras for applications like robotics or AR/VR, but it is incremental as it builds on existing learning-based methods by incorporating 3D information.
The paper tackled denoising Time-of-Flight camera data affected by noise and Multi-Path-Interference by proposing an iterative 3D approach using ray-aligned convolutions and self-training on unlabeled real-world data, achieving state-of-the-art performance on multiple datasets.
Time-of-Flight (ToF) cameras are subject to high levels of noise and distortions due to Multi-Path-Interference (MPI). While recent research showed that 2D neural networks are able to outperform previous traditional State-of-the-Art (SOTA) methods on denoising ToF-Data, little research on learning-based approaches has been done to make direct use of the 3D information present in depth images. In this paper, we propose an iterative denoising approach operating in 3D space, that is designed to learn on 2.5D data by enabling 3D point convolutions to correct the points' positions along the view direction. As labeled real world data is scarce for this task, we further train our network with a self-training approach on unlabeled real world data to account for real world statistics. We demonstrate that our method is able to outperform SOTA methods on several datasets, including two real world datasets and a new large-scale synthetic data set introduced in this paper.