X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance
This work addresses a crucial task in computer vision and graphics for generating stylized 3D assets from text, though it appears incremental by improving upon existing CLIP-based approaches.
The paper tackles the problem of text-driven 3D stylization, where prior methods suffer from slow convergence and inaccurate results due to lack of textual guidance during attribute prediction; it introduces X-Mesh with a Text-guided Dynamic Attention Module, achieving faster convergence and outperforming state-of-the-art methods in experiments.
Text-driven 3D stylization is a complex and crucial task in the fields of computer vision (CV) and computer graphics (CG), aimed at transforming a bare mesh to fit a target text. Prior methods adopt text-independent multilayer perceptrons (MLPs) to predict the attributes of the target mesh with the supervision of CLIP loss. However, such text-independent architecture lacks textual guidance during predicting attributes, thus leading to unsatisfactory stylization and slow convergence. To address these limitations, we present X-Mesh, an innovative text-driven 3D stylization framework that incorporates a novel Text-guided Dynamic Attention Module (TDAM). The TDAM dynamically integrates the guidance of the target text by utilizing text-relevant spatial and channel-wise attentions during vertex feature extraction, resulting in more accurate attribute prediction and faster convergence speed. Furthermore, existing works lack standard benchmarks and automated metrics for evaluation, often relying on subjective and non-reproducible user studies to assess the quality of stylized 3D assets. To overcome this limitation, we introduce a new standard text-mesh benchmark, namely MIT-30, and two automated metrics, which will enable future research to achieve fair and objective comparisons. Our extensive qualitative and quantitative experiments demonstrate that X-Mesh outperforms previous state-of-the-art methods.