VINS: Visual Search for Mobile User Interface Design
This addresses a challenge for mobile interface designers by providing a tool to find design inspiration and compare alternatives, though it is incremental as it builds on existing object-detection methods.
The paper tackles the problem of finding similar mobile UI design examples by introducing VINS, a visual search framework that uses UI images as input to retrieve visually similar designs, achieving a mean Average Precision of 76.39% for UI detection.
Searching for relative mobile user interface (UI) design examples can aid interface designers in gaining inspiration and comparing design alternatives. However, finding such design examples is challenging, especially as current search systems rely on only text-based queries and do not consider the UI structure and content into account. This paper introduces VINS, a visual search framework, that takes as input a UI image (wireframe, high-fidelity) and retrieves visually similar design examples. We first survey interface designers to better understand their example finding process. We then develop a large-scale UI dataset that provides an accurate specification of the interface's view hierarchy (i.e., all the UI components and their specific location). By utilizing this dataset, we propose an object-detection based image retrieval framework that models the UI context and hierarchical structure. The framework achieves a mean Average Precision of 76.39\% for the UI detection and high performance in querying similar UI designs.