ImageEye: Batch Image Processing Using Program Synthesis
This addresses the need for more precise and automated image editing tools for users in fields like photography or design, though it is incremental as it builds on existing synthesis and neuro-symbolic methods.
The paper tackles the problem of batch image processing with fine-grained edits to individual objects, achieving automation of 96% of tasks in an evaluation on 50 image editing tasks.
This paper presents a new synthesis-based approach for batch image processing. Unlike existing tools that can only apply global edits to the entire image, our method can apply fine-grained edits to individual objects within the image. For example, our method can selectively blur or crop specific objects that have a certain property. To facilitate such fine-grained image editing tasks, we propose a neuro-symbolic domain-specific language (DSL) that combines pre-trained neural networks for image classification with other language constructs that enable symbolic reasoning. Our method can automatically learn programs in this DSL from user demonstrations by utilizing a novel synthesis algorithm. We have implemented the proposed technique in a tool called ImageEye and evaluated it on 50 image editing tasks. Our evaluation shows that ImageEye is able to automate 96% of these tasks.