PlückeRF: A Line-based 3D Representation for Few-view Reconstruction
This work addresses the challenge of enhancing 3D reconstruction from limited views for applications in computer vision and graphics, representing an incremental improvement over existing methods.
The paper tackles the problem of few-view 3D reconstruction by proposing a line-based representation called PlückeRF, which connects 3D features with pixel rays to better leverage multi-view information, resulting in improved reconstruction quality over triplane and state-of-the-art feedforward methods.
Feed-forward 3D reconstruction methods aim to predict the 3D structure of a scene directly from input images, providing a faster alternative to per-scene optimization approaches. Significant progress has been made in single-view and few-view reconstruction using learned priors that infer object shape and appearance, even for unobserved regions. However, there is substantial potential to enhance these methods by better leveraging information from multiple views when available. To address this, we propose a few-view reconstruction model that more effectively harnesses multi-view information. Our approach introduces a simple mechanism that connects the 3D representation with pixel rays from the input views, allowing for preferential sharing of information between nearby 3D locations and between 3D locations and nearby pixel rays. We achieve this by defining the 3D representation as a set of structured, feature-augmented lines; the PlückeRF representation. Using this representation, we demonstrate improvements in reconstruction quality over the equivalent triplane representation and state-of-the-art feedforward reconstruction methods.