HumanOrbit: 3D Human Reconstruction as 360° Orbit Generation
This addresses the challenge of consistent multi-view synthesis and 3D reconstruction for human images, which is incremental by building on video diffusion models.
The paper tackles the problem of generating a full 360° orbit video and reconstructing a 3D textured mesh from a single input image of a person, achieving superior completeness and fidelity compared to state-of-the-art baselines.
We present a method for generating a full 360° orbit video around a person from a single input image. Existing methods typically adapt image-based diffusion models for multi-view synthesis, but yield inconsistent results across views and with the original identity. In contrast, recent video diffusion models have demonstrated their ability in generating photorealistic results that align well with the given prompts. Inspired by these results, we propose HumanOrbit, a video diffusion model for multi-view human image generation. Our approach enables the model to synthesize continuous camera rotations around the subject, producing geometrically consistent novel views while preserving the appearance and identity of the person. Using the generated multi-view frames, we further propose a reconstruction pipeline that recovers a textured mesh of the subject. Experimental results validate the effectiveness of HumanOrbit for multi-view image generation and that the reconstructed 3D models exhibit superior completeness and fidelity compared to those from state-of-the-art baselines.