ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation
This addresses the need for precise 3D orientation control in image generation for applications like design and visualization, representing a novel advancement beyond 2D spatial grounding.
The authors tackled the problem of controlling 3D orientation in text-to-image generation, which previous methods lacked, and achieved state-of-the-art performance by outperforming existing methods in quantitative metrics and user studies.
We introduce ORIGEN, the first zero-shot method for 3D orientation grounding in text-to-image generation across multiple objects and diverse categories. While previous work on spatial grounding in image generation has mainly focused on 2D positioning, it lacks control over 3D orientation. To address this, we propose a reward-guided sampling approach using a pretrained discriminative model for 3D orientation estimation and a one-step text-to-image generative flow model. While gradient-ascent-based optimization is a natural choice for reward-based guidance, it struggles to maintain image realism. Instead, we adopt a sampling-based approach using Langevin dynamics, which extends gradient ascent by simply injecting random noise--requiring just a single additional line of code. Additionally, we introduce adaptive time rescaling based on the reward function to accelerate convergence. Our experiments show that ORIGEN outperforms both training-based and test-time guidance methods across quantitative metrics and user studies.