Fast and Accurate: Video Enhancement using Sparse Depth
This addresses the problem of slow and less accurate video enhancement for applications in autonomous agents, offering a fast and high-quality solution.
The paper tackles video enhancement tasks like super-resolution and denoising by proposing a framework that uses sparse depth and IMU data for flow estimation, achieving speedups of 1.78x to 187.41x and quality improvements of 0.42 dB to 6.70 dB over competing methods.
This paper presents a general framework to build fast and accurate algorithms for video enhancement tasks such as super-resolution, deblurring, and denoising. Essential to our framework is the realization that the accuracy, rather than the density, of pixel flows is what is required for high-quality video enhancement. Most of prior works take the opposite approach: they estimate dense (per-pixel)-but generally less robust-flows, mostly using computationally costly algorithms. Instead, we propose a lightweight flow estimation algorithm; it fuses the sparse point cloud data and (even sparser and less reliable) IMU data available in modern autonomous agents to estimate the flow information. Building on top of the flow estimation, we demonstrate a general framework that integrates the flows in a plug-and-play fashion with different task-specific layers. Algorithms built in our framework achieve 1.78x - 187.41x speedup while providing a 0.42 dB - 6.70 dB quality improvement over competing methods.