Ray Conditioning: Trading Photo-consistency for Photo-realism in Multi-view Image Generation
This addresses the problem of geometry artifacts and loss of details in multi-view image editing for real images, offering a geometry-free alternative.
The paper tackles the trade-off between photo-consistency and photo-realism in multi-view image generation for tasks like viewpoint editing, proposing ray conditioning to achieve state-of-the-art photo-realism and identity consistency.
Multi-view image generation attracts particular attention these days due to its promising 3D-related applications, e.g., image viewpoint editing. Most existing methods follow a paradigm where a 3D representation is first synthesized, and then rendered into 2D images to ensure photo-consistency across viewpoints. However, such explicit bias for photo-consistency sacrifices photo-realism, causing geometry artifacts and loss of fine-scale details when these methods are applied to edit real images. To address this issue, we propose ray conditioning, a geometry-free alternative that relaxes the photo-consistency constraint. Our method generates multi-view images by conditioning a 2D GAN on a light field prior. With explicit viewpoint control, state-of-the-art photo-realism and identity consistency, our method is particularly suited for the viewpoint editing task.