Robust Optical Flow Estimation of Double-Layer Images under Transparency or Reflection
This addresses a challenging and under-investigated problem in computer vision for applications like robotics or augmented reality, but it is incremental as it builds on existing optical flow techniques.
The paper tackles the problem of optical flow estimation in images with transparency or reflection, where conventional methods fail due to invalid brightness constancy constraints, and proposes a method that jointly estimates optical flow and separates image layers, achieving efficacy confirmed on synthetic and real data.
This paper deals with a challenging, frequently encountered, yet not properly investigated problem in two-frame optical flow estimation. That is, the input frames are compounds of two imaging layers -- one desired background layer of the scene, and one distracting, possibly moving layer due to transparency or reflection. In this situation, the conventional brightness constancy constraint -- the cornerstone of most existing optical flow methods -- will no longer be valid. In this paper, we propose a robust solution to this problem. The proposed method performs both optical flow estimation, and image layer separation. It exploits a generalized double-layer brightness consistency constraint connecting these two tasks, and utilizes the priors for both of them. Experiments on both synthetic data and real images have confirmed the efficacy of the proposed method. To the best of our knowledge, this is the first attempt towards handling generic optical flow fields of two-frame images containing transparency or reflection.