GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing
This addresses the need for more realistic and compatible object placement in image compositing for users in graphics and design, representing an incremental improvement over prior methods.
The paper tackles the problem of compositing-aware object search by proposing GALA, a method that models geometry and lighting compatibility to find compatible foreground objects for image compositing, achieving state-of-the-art results on the CAIS dataset and generalizing well on large-scale datasets like Pixabay and Open Images.
Compositing-aware object search aims to find the most compatible objects for compositing given a background image and a query bounding box. Previous works focus on learning compatibility between the foreground object and background, but fail to learn other important factors from large-scale data, i.e. geometry and lighting. To move a step further, this paper proposes GALA (Geometry-and-Lighting-Aware), a generic foreground object search method with discriminative modeling on geometry and lighting compatibility for open-world image compositing. Remarkably, it achieves state-of-the-art results on the CAIS dataset and generalizes well on large-scale open-world datasets, i.e. Pixabay and Open Images. In addition, our method can effectively handle non-box scenarios, where users only provide background images without any input bounding box. A web demo (see supplementary materials) is built to showcase applications of the proposed method for compositing-aware search and automatic location/scale prediction for the foreground object.