Self-Assessed Generation: Trustworthy Label Generation for Optical Flow and Stereo Matching in Real-world
This addresses the problem of high dataset costs and limited self-supervised methods for researchers and practitioners in computer vision, though it appears incremental as it builds on existing self-supervised approaches without altering core methods.
The paper tackles the challenge of generalizing optical flow and stereo methods to real-world scenarios by proposing Self-Assessed Generation (SAG), a self-supervised framework that uses reconstruction techniques and confidence quantification to generate datasets, resulting in improved generalization performance on mainstream datasets.
A significant challenge facing current optical flow and stereo methods is the difficulty in generalizing them well to the real world. This is mainly due to the high costs required to produce datasets, and the limitations of existing self-supervised methods on fuzzy results and complex model training problems. To address the above challenges, we propose a unified self-supervised generalization framework for optical flow and stereo tasks: Self-Assessed Generation (SAG). Unlike previous self-supervised methods, SAG is data-driven, using advanced reconstruction techniques to construct a reconstruction field from RGB images and generate datasets based on it. Afterward, we quantified the confidence level of the generated results from multiple perspectives, such as reconstruction field distribution, geometric consistency, and structural similarity, to eliminate inevitable defects in the generation process. We also designed a 3D flight foreground automatic rendering pipeline in SAG to encourage the network to learn occlusion and motion foreground. Experimentally, because SAG does not involve changes to methods or loss functions, it can directly self-supervised train the state-of-the-art deep networks, greatly improving the generalization performance of self-supervised methods on current mainstream optical flow and stereo-matching datasets. Compared to previous training modes, SAG is more generalized, cost-effective, and accurate.