Retargeting Visual Data with Deformation Fields
This work addresses the limitation of existing content-aware editing methods for broader visual data formats, offering a more flexible solution for applications in image processing and computer graphics.
The paper tackles the problem of content-aware resizing and editing of visual data by proposing a neural network that learns deformation fields, generalizing beyond traditional seam carving methods. It achieves better retargeting results across images, 3D scenes, and meshes compared to prior approaches.
Seam carving is an image editing method that enable content-aware resizing, including operations like removing objects. However, the seam-finding strategy based on dynamic programming or graph-cut limits its applications to broader visual data formats and degrees of freedom for editing. Our observation is that describing the editing and retargeting of images more generally by a displacement field yields a generalisation of content-aware deformations. We propose to learn a deformation with a neural network that keeps the output plausible while trying to deform it only in places with low information content. This technique applies to different kinds of visual data, including images, 3D scenes given as neural radiance fields, or even polygon meshes. Experiments conducted on different visual data show that our method achieves better content-aware retargeting compared to previous methods.