Optimizing User Interface Layouts via Gradient Descent
This addresses the challenge of UI design automation for mobile interfaces, but it is incremental as it extends prior work with an expanded interaction space.
The paper tackles the problem of automating user interface layout optimization for mobile UIs by using gradient descent on a neural network model, resulting in up to 9.2% improvements in predicted and experimentally confirmed task performance.
Automating parts of the user interface (UI) design process has been a longstanding challenge. We present an automated technique for optimizing the layouts of mobile UIs. Our method uses gradient descent on a neural network model of task performance with respect to the model's inputs to make layout modifications that result in improved predicted error rates and task completion times. We start by extending prior work on neural network based performance prediction to 2-dimensional mobile UIs with an expanded interaction space. We then apply our method to two UIs, including one that the model had not been trained on, to discover layout alternatives with significantly improved predicted performance. Finally, we confirm these predictions experimentally, showing improvements up to 9.2 percent in the optimized layouts. This demonstrates the algorithm's efficacy in improving the task performance of a layout, and its ability to generalize and improve layouts of new interfaces.