DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Diffusion Model
This addresses the problem of unreliable correlation and accumulated inaccuracy in scene flow estimation for computer vision applications, offering a plug-and-play module for existing networks.
The paper tackles scene flow estimation by proposing DifFlow3D, a diffusion-based network that improves correlation robustness and handles challenging cases like dynamics and noise, achieving state-of-the-art performance with 24.0% and 29.1% EPE3D reductions on FlyingThings3D and KITTI 2015 datasets, and an unprecedented 0.0078m EPE3D accuracy on KITTI.
Scene flow estimation, which aims to predict per-point 3D displacements of dynamic scenes, is a fundamental task in the computer vision field. However, previous works commonly suffer from unreliable correlation caused by locally constrained searching ranges, and struggle with accumulated inaccuracy arising from the coarse-to-fine structure. To alleviate these problems, we propose a novel uncertainty-aware scene flow estimation network (DifFlow3D) with the diffusion probabilistic model. Iterative diffusion-based refinement is designed to enhance the correlation robustness and resilience to challenging cases, e.g. dynamics, noisy inputs, repetitive patterns, etc. To restrain the generation diversity, three key flow-related features are leveraged as conditions in our diffusion model. Furthermore, we also develop an uncertainty estimation module within diffusion to evaluate the reliability of estimated scene flow. Our DifFlow3D achieves state-of-the-art performance, with 24.0% and 29.1% EPE3D reduction respectively on FlyingThings3D and KITTI 2015 datasets. Notably, our method achieves an unprecedented millimeter-level accuracy (0.0078m in EPE3D) on the KITTI dataset. Additionally, our diffusion-based refinement paradigm can be readily integrated as a plug-and-play module into existing scene flow networks, significantly increasing their estimation accuracy. Codes are released at https://github.com/IRMVLab/DifFlow3D.