LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoning
This addresses the limitation of conventional depth estimation for single-view 3D reasoning, offering a more comprehensive solution for computer vision tasks involving objects and scenes.
The paper tackles the problem of reasoning about unseen 3D geometry from a single image by introducing LaRI, which models multiple surfaces using layered ray intersections, enabling complete and efficient geometric reasoning. It achieves comparable object-level results to a large generative model with only 4% of its training data and 17% of its parameters, and performs scene-level occluded geometry reasoning in one feed-forward pass.
We present layered ray intersections (LaRI), a new method for unseen geometry reasoning from a single image. Unlike conventional depth estimation that is limited to the visible surface, LaRI models multiple surfaces intersected by the camera rays using layered point maps. Benefiting from the compact and layered representation, LaRI enables complete, efficient, and view-aligned geometric reasoning to unify object- and scene-level tasks. We further propose to predict the ray stopping index, which identifies valid intersecting pixels and layers from LaRI's output. We build a complete training data generation pipeline for synthetic and real-world data, including 3D objects and scenes, with necessary data cleaning steps and coordination between rendering engines. As a generic method, LaRI's performance is validated in two scenarios: It yields comparable object-level results to the recent large generative model using 4% of its training data and 17% of its parameters. Meanwhile, it achieves scene-level occluded geometry reasoning in only one feed-forward.