Accurate and efficient video de-fencing using convolutional neural networks and temporal information
This work addresses video de-fencing for photographers and computer vision applications, but it is incremental as it builds on existing methods with specific enhancements.
The paper tackles the problem of removing fences from videos by improving fence segmentation accuracy and handling camera or object motion, achieving state-of-the-art performance in both segmentation and content recovery.
De-fencing is to eliminate the captured fence on an image or a video, providing a clear view of the scene. It has been applied for many purposes including assisting photographers and improving the performance of computer vision algorithms such as object detection and recognition. However, the state-of-the-art de-fencing methods have limited performance caused by the difficulty of fence segmentation and also suffer from the motion of the camera or objects. To overcome these problems, we propose a novel method consisting of segmentation using convolutional neural networks and a fast/robust recovery algorithm. The segmentation algorithm using convolutional neural network achieves significant improvement in the accuracy of fence segmentation. The recovery algorithm using optical flow produces plausible de-fenced images and videos. The proposed method is experimented on both our diverse and complex dataset and publicly available datasets. The experimental results demonstrate that the proposed method achieves the state-of-the-art performance for both segmentation and content recovery.