DuetSVG: Unified Multimodal SVG Generation with Internal Visual Guidance
This work addresses the challenge of generating high-quality SVGs for applications requiring visually appealing and semantically accurate graphics, representing an incremental advance in multimodal generation techniques.
The paper tackles the problem of generating Scalable Vector Graphics (SVGs) from vision-language models, which often produce poor visual and geometric results due to lack of visual signals during decoding, and introduces DuetSVG, a unified multimodal model that jointly generates image and SVG tokens, achieving improved performance over existing methods.
Recent vision-language model (VLM)-based approaches have achieved impressive results on SVG generation. However, because they generate only text and lack visual signals during decoding, they often struggle with complex semantics and fail to produce visually appealing or geometrically coherent SVGs. We introduce DuetSVG, a unified multimodal model that jointly generates image tokens and corresponding SVG tokens in an end-to-end manner. DuetSVG is trained on both image and SVG datasets. At inference, we apply a novel test-time scaling strategy that leverages the model's native visual predictions as guidance to improve SVG decoding quality. Extensive experiments show that our method outperforms existing methods, producing visually faithful, semantically aligned, and syntactically clean SVGs across a wide range of applications.