Context-Consistent Semantic Image Editing with Style-Preserved Modulation
This work addresses style inconsistency issues in semantic image editing for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles the problem of style discrepancy in semantic image editing by proposing a style-preserved modulation (SPM) method that incorporates contextual styles and semantic layouts, achieving context-consistent results and significantly alleviating unpleasant boundaries.
Semantic image editing utilizes local semantic label maps to generate the desired content in the edited region. A recent work borrows SPADE block to achieve semantic image editing. However, it cannot produce pleasing results due to style discrepancy between the edited region and surrounding pixels. We attribute this to the fact that SPADE only uses an image-independent local semantic layout but ignores the image-specific styles included in the known pixels. To address this issue, we propose a style-preserved modulation (SPM) comprising two modulations processes: The first modulation incorporates the contextual style and semantic layout, and then generates two fused modulation parameters. The second modulation employs the fused parameters to modulate feature maps. By using such two modulations, SPM can inject the given semantic layout while preserving the image-specific context style. Moreover, we design a progressive architecture for generating the edited content in a coarse-to-fine manner. The proposed method can obtain context-consistent results and significantly alleviate the unpleasant boundary between the generated regions and the known pixels.