Automatic Foreground Extraction from Imperfect Backgrounds using Multi-Agent Consensus Equilibrium
This addresses the need for automated foreground extraction in applications like filming and surveillance, reducing reliance on manual interventions, though it appears incremental as it integrates existing techniques.
The paper tackles the problem of automatically extracting high-quality foreground objects from scenes with imperfect static backgrounds, achieving significantly better results than state-of-the-art methods in background subtraction, video segmentation, and alpha matting.
Extracting accurate foreground objects from a scene is an essential step for many video applications. Traditional background subtraction algorithms can generate coarse estimates, but generating high quality masks requires professional softwares with significant human interventions, e.g., providing trimaps or labeling key frames. We propose an automatic foreground extraction method in applications where a static but imperfect background is available. Examples include filming and surveillance where the background can be captured before the objects enter the scene or after they leave the scene. Our proposed method is very robust and produces significantly better estimates than state-of-the-art background subtraction, video segmentation and alpha matting methods. The key innovation of our method is a novel information fusion technique. The fusion framework allows us to integrate the individual strengths of alpha matting, background subtraction and image denoising to produce an overall better estimate. Such integration is particularly important when handling complex scenes with imperfect background. We show how the framework is developed, and how the individual components are built. Extensive experiments and ablation studies are conducted to evaluate the proposed method.