A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D
This work addresses the lack of objective evaluation for text-to-3D generation artifacts, which is a problem for researchers and practitioners in 3D content creation, though it is incremental as it builds on existing SDS methods.
The paper tackles the problem of artifacts in text-to-3D generation using score distillation sampling (SDS), such as the Janus problem and misalignment, by proposing quantitative evaluation metrics validated with human ratings and analyzing failure cases, resulting in a novel baseline model that achieves state-of-the-art performance on these metrics while addressing the artifacts.
The development of generative models that create 3D content from a text prompt has made considerable strides thanks to the use of the score distillation sampling (SDS) method on pre-trained diffusion models for image generation. However, the SDS method is also the source of several artifacts, such as the Janus problem, the misalignment between the text prompt and the generated 3D model, and 3D model inaccuracies. While existing methods heavily rely on the qualitative assessment of these artifacts through visual inspection of a limited set of samples, in this work we propose more objective quantitative evaluation metrics, which we cross-validate via human ratings, and show analysis of the failure cases of the SDS technique. We demonstrate the effectiveness of this analysis by designing a novel computationally efficient baseline model that achieves state-of-the-art performance on the proposed metrics while addressing all the above-mentioned artifacts.