DOne: Decoupling Structure and Rendering for High-Fidelity Design-to-Code Generation
This work solves the problem of generating high-fidelity code from designs for developers and designers, but it is incremental as it builds on existing Vision Language Models with novel decoupling techniques.
The paper tackles the problem of design-to-code generation by addressing the holistic bottleneck in Vision Language Models, which often leads to layout distortions or generic placeholders; the result is that DOne outperforms existing methods with over 10% improvement in GPT Score and achieves a 3 times productivity gain in human evaluations.
While Vision Language Models (VLMs) have shown promise in Design-to-Code generation, they suffer from a "holistic bottleneck-failing to reconcile high-level structural hierarchy with fine-grained visual details, often resulting in layout distortions or generic placeholders. To bridge this gap, we propose DOne, an end-to-end framework that decouples structure understanding from element rendering. DOne introduces (1) a learned layout segmentation module to decompose complex designs, avoiding the limitations of heuristic cropping; (2) a specialized hybrid element retriever to handle the extreme aspect ratios and densities of UI components; and (3) a schema-guided generation paradigm that bridges layout and code. To rigorously assess performance, we introduce HiFi2Code, a benchmark featuring significantly higher layout complexity than existing datasets. Extensive evaluations on the HiFi2Code demonstrate that DOne outperforms exiting methods in both high-level visual similarity (e.g., over 10% in GPT Score) and fine-grained element alignment. Human evaluations confirm a 3 times productivity gain with higher visual fidelity.