DreamCS: Geometry-Aware Text-to-3D Generation with Unpaired 3D Reward Supervision
This addresses the challenge of generating high-quality 3D content from text for applications in gaming, VR, and design, though it is incremental as it builds on existing text-to-3D pipelines.
The paper tackles the problem of text-to-3D generation often producing 3D assets misaligned with human preferences by introducing DreamCS, a framework that integrates a 3D reward model trained on unpaired data, resulting in improved geometric faithfulness and human preference alignment compared to prior methods.
While text-to-3D generation has attracted growing interest, existing methods often struggle to produce 3D assets that align well with human preferences. Current preference alignment techniques for 3D content typically rely on hardly-collected preference-paired multi-view 2D images to train 2D reward models, when then guide 3D generation -- leading to geometric artifacts due to their inherent 2D bias. To address these limitations, we construct 3D-MeshPref, the first large-scale unpaired 3D preference dataset, featuring diverse 3D meshes annotated by a large language model and refined by human evaluators. We then develop RewardCS, the first reward model trained directly on unpaired 3D-MeshPref data using a novel Cauchy-Schwarz divergence objective, enabling effective learning of human-aligned 3D geometric preferences without requiring paired comparisons. Building on this, we propose DreamCS, a unified framework that integrates RewardCS into text-to-3D pipelines -- enhancing both implicit and explicit 3D generation with human preference feedback. Extensive experiments show DreamCS outperforms prior methods, producing 3D assets that are both geometrically faithful and human-preferred. Code and models will be released publicly.