Semi-supervised Parametric Real-world Image Harmonization
This addresses the generalization issue in image harmonization for real-world composites, though it appears incremental as it builds on existing parametric methods with a new training approach.
The paper tackles the problem of image harmonization where foreground and background from different images have appearance mismatches, proposing a semi-supervised training strategy that learns complex local harmonization from unpaired real composites, outperforming previous work on benchmarks and in a user study.
Learning-based image harmonization techniques are usually trained to undo synthetic random global transformations applied to a masked foreground in a single ground truth photo. This simulated data does not model many of the important appearance mismatches (illumination, object boundaries, etc.) between foreground and background in real composites, leading to models that do not generalize well and cannot model complex local changes. We propose a new semi-supervised training strategy that addresses this problem and lets us learn complex local appearance harmonization from unpaired real composites, where foreground and background come from different images. Our model is fully parametric. It uses RGB curves to correct the global colors and tone and a shading map to model local variations. Our method outperforms previous work on established benchmarks and real composites, as shown in a user study, and processes high-resolution images interactively.