Single-View View Synthesis with Multiplane Images
This work addresses the challenging problem of generating novel views from a single image, which has broad applications in computer vision and graphics, though it builds on existing multiplane image methods.
The paper tackles single-view view synthesis by predicting a multiplane image from a single input image, using scale-invariant supervision from online video, and demonstrates applicability across datasets with reasonable depth maps and content filling behind object edges.
A recent strand of work in view synthesis uses deep learning to generate multiplane images (a camera-centric, layered 3D representation) given two or more input images at known viewpoints. We apply this representation to single-view view synthesis, a problem which is more challenging but has potentially much wider application. Our method learns to predict a multiplane image directly from a single image input, and we introduce scale-invariant view synthesis for supervision, enabling us to train on online video. We show this approach is applicable to several different datasets, that it additionally generates reasonable depth maps, and that it learns to fill in content behind the edges of foreground objects in background layers. Project page at https://single-view-mpi.github.io/.