Generative Modeling in Structural-Hankel Domain for Color Image Inpainting
This addresses the problem of efficient sample usage in image inpainting for computer vision applications, but it appears incremental as it builds on existing score-based generative models with a novel matrix construction.
The paper tackled color image inpainting by proposing a low-rank structural-Hankel matrices-assisted score-based generative model (SHGM) that requires only ten or fewer samples, achieving remarkable performance and diversity in experimental results.
In recent years, some researchers focused on using a single image to obtain a large number of samples through multi-scale features. This study intends to a brand-new idea that requires only ten or even fewer samples to construct the low-rank structural-Hankel matrices-assisted score-based generative model (SHGM) for color image inpainting task. During the prior learning process, a certain amount of internal-middle patches are firstly extracted from several images and then the structural-Hankel matrices are constructed from these patches. To better apply the score-based generative model to learn the internal statistical distribution within patches, the large-scale Hankel matrices are finally folded into the higher dimensional tensors for prior learning. During the iterative inpainting process, SHGM views the inpainting problem as a conditional generation procedure in low-rank environment. As a result, the intermediate restored image is acquired by alternatively performing the stochastic differential equation solver, alternating direction method of multipliers, and data consistency steps. Experimental results demonstrated the remarkable performance and diversity of SHGM.