Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Models
This addresses the challenge for app designers and developers in reducing effort and expertise needed to create UI mock-ups from text descriptions, though it appears incremental as it builds on existing deep-learning approaches.
The paper tackled the problem of translating high-level text descriptions into low-fidelity UI mock-ups by introducing three deep-learning techniques, including retrieval-based and generative methods, and found that professional designers responded positively to their potential for assisting the design process.
The design process of user interfaces (UIs) often begins with articulating high-level design goals. Translating these high-level design goals into concrete design mock-ups, however, requires extensive effort and UI design expertise. To facilitate this process for app designers and developers, we introduce three deep-learning techniques to create low-fidelity UI mock-ups from a natural language phrase that describes the high-level design goal (e.g. "pop up displaying an image and other options"). In particular, we contribute two retrieval-based methods and one generative method, as well as pre-processing and post-processing techniques to ensure the quality of the created UI mock-ups. We quantitatively and qualitatively compare and contrast each method's ability in suggesting coherent, diverse and relevant UI design mock-ups. We further evaluate these methods with 15 professional UI designers and practitioners to understand each method's advantages and disadvantages. The designers responded positively to the potential of these methods for assisting the design process.