SteerX: Creating Any Camera-Free 3D and 4D Scenes with Geometric Steering
This addresses alignment issues in 3D/4D scene generation for applications like video synthesis and reconstruction, but it appears incremental as it builds on existing methods with new steering techniques.
The paper tackles the problem of subtle misalignments in 3D/4D scene generation by introducing SteerX, a zero-shot inference-time steering method that unifies scene reconstruction into generation, resulting in improved geometric alignment as demonstrated through extensive experiments.
Recent progress in 3D/4D scene generation emphasizes the importance of physical alignment throughout video generation and scene reconstruction. However, existing methods improve the alignment separately at each stage, making it difficult to manage subtle misalignments arising from another stage. Here, we present SteerX, a zero-shot inference-time steering method that unifies scene reconstruction into the generation process, tilting data distributions toward better geometric alignment. To this end, we introduce two geometric reward functions for 3D/4D scene generation by using pose-free feed-forward scene reconstruction models. Through extensive experiments, we demonstrate the effectiveness of SteerX in improving 3D/4D scene generation.