Sketch2code: Generating a website from a paper mockup
This work addresses the time-consuming task of converting wireframes into code for developers, though it is incremental as it builds on existing automation ideas.
The paper tackles the problem of automating website development from paper mockups by presenting two approaches: classical computer vision and a novel deep semantic segmentation method, with the deep learning approach outperforming the classical one in evaluation.
An early stage of developing user-facing applications is creating a wireframe to layout the interface. Once a wireframe has been created it is given to a developer to implement in code. Developing boiler plate user interface code is time consuming work but still requires an experienced developer. In this dissertation we present two approaches which automates this process, one using classical computer vision techniques, and another using a novel application of deep semantic segmentation networks. We release a dataset of websites which can be used to train and evaluate these approaches. Further, we have designed a novel evaluation framework which allows empirical evaluation by creating synthetic sketches. Our evaluation illustrates that our deep learning approach outperforms our classical computer vision approach and we conclude that deep learning is the most promising direction for future research.