E-Commerce Inpainting with Mask Guidance in Controlnet for Reducing Overcompletion
This addresses a core pain point in e-commerce image generation for businesses, but it is incremental as it builds on existing diffusion models and ControlNet.
The paper tackles the problem of overcompletion in e-commerce image generation using diffusion models, where product features are difficult to maintain, and proposes solutions including an instance mask fine-tuned inpainting model and a train-free mask guidance approach, achieving promising results in practical applications.
E-commerce image generation has always been one of the core demands in the e-commerce field. The goal is to restore the missing background that matches the main product given. In the post-AIGC era, diffusion models are primarily used to generate product images, achieving impressive results. This paper systematically analyzes and addresses a core pain point in diffusion model generation: overcompletion, which refers to the difficulty in maintaining product features. We propose two solutions: 1. Using an instance mask fine-tuned inpainting model to mitigate this phenomenon; 2. Adopting a train-free mask guidance approach, which incorporates refined product masks as constraints when combining ControlNet and UNet to generate the main product, thereby avoiding overcompletion of the product. Our method has achieved promising results in practical applications and we hope it can serve as an inspiring technical report in this field.