Improve Deep Image Inpainting by Emphasizing the Complexity of Missing Regions
This work addresses the challenge of optimizing deep inpainting models for researchers and practitioners, though it is incremental as it enhances existing methods rather than introducing a new paradigm.
The paper tackles the problem of limited performance gains in deep image inpainting by using classical image complexity metrics to guide batch selection during training, resulting in improved inpainting performance across multiple models and datasets.
Deep image inpainting research mainly focuses on constructing various neural network architectures or imposing novel optimization objectives. However, on the one hand, building a state-of-the-art deep inpainting model is an extremely complex task, and on the other hand, the resulting performance gains are sometimes very limited. We believe that besides the frameworks of inpainting models, lightweight traditional image processing techniques, which are often overlooked, can actually be helpful to these deep models. In this paper, we enhance the deep image inpainting models with the help of classical image complexity metrics. A knowledge-assisted index composed of missingness complexity and forward loss is presented to guide the batch selection in the training procedure. This index helps find samples that are more conducive to optimization in each iteration and ultimately boost the overall inpainting performance. The proposed approach is simple and can be plugged into many deep inpainting models by changing only a few lines of code. We experimentally demonstrate the improvements for several recently developed image inpainting models on various datasets.