Generating Synthetic Stereo Datasets using 3D Gaussian Splatting and Expert Knowledge Transfer
This work addresses the need for low-cost, high-fidelity dataset creation for deep stereo models, though it is incremental as it builds on existing 3D Gaussian Splatting and expert knowledge transfer techniques.
The paper tackles the problem of generating synthetic stereo datasets by introducing a 3D Gaussian Splatting-based pipeline as an efficient alternative to NeRF-based methods, achieving competitive performance in zero-shot generalization benchmarks when fine-tuning stereo models on these datasets.
In this paper, we introduce a 3D Gaussian Splatting (3DGS)-based pipeline for stereo dataset generation, offering an efficient alternative to Neural Radiance Fields (NeRF)-based methods. To obtain useful geometry estimates, we explore utilizing the reconstructed geometry from the explicit 3D representations as well as depth estimates from the FoundationStereo model in an expert knowledge transfer setup. We find that when fine-tuning stereo models on 3DGS-generated datasets, we demonstrate competitive performance in zero-shot generalization benchmarks. When using the reconstructed geometry directly, we observe that it is often noisy and contains artifacts, which propagate noise to the trained model. In contrast, we find that the disparity estimates from FoundationStereo are cleaner and consequently result in a better performance on the zero-shot generalization benchmarks. Our method highlights the potential for low-cost, high-fidelity dataset creation and fast fine-tuning for deep stereo models. Moreover, we also reveal that while the latest Gaussian Splatting based methods have achieved superior performance on established benchmarks, their robustness falls short in challenging in-the-wild settings warranting further exploration.