Unsupervised Enhancement of Real-World Depth Images Using Tri-Cycle GAN
This addresses a challenge for computer vision applications using low-cost depth sensors, but it is incremental as it builds on existing Cycle-GAN methods.
The paper tackled the problem of enhancing low-quality real-world depth images from a low-cost sensor without clean ground-truth, using an unsupervised domain-translation approach with modifications to Cycle-GAN, resulting in dramatic improvements over the original method both visually and quantitatively.
Low quality depth poses a considerable challenge to computer vision algorithms. In this work we aim to enhance highly degraded, real-world depth images acquired by a low-cost sensor, for which an analytical noise model is unavailable. In the absence of clean ground-truth, we approach the task as an unsupervised domain-translation between the low-quality sensor domain and a high-quality sensor domain, represented using two unpaired training sets. We employ the highly-successful Cycle-GAN to this task, but find it to perform poorly in this case. Identifying the sources of the failure, we introduce several modifications to the framework, including a larger generator architecture, depth-specific losses that take into account missing pixels, and a novel Tri-Cycle loss which promotes information-preservation while addressing the asymmetry between the domains. We show that the resulting framework dramatically improves over the original Cycle-GAN both visually and quantitatively, extending its applicability to more challenging and asymmetric translation tasks.