Towards Recommending Usability Improvements with Multimodal Large Language Models
This addresses the resource-intensive nature of usability evaluation for smaller organizations, offering a practical alternative, though it appears incremental as it builds on existing LLM capabilities.
The paper tackles the problem of automating usability evaluation of user interfaces by formulating it as a recommendation task using multimodal LLMs to rank usability issues by severity, with results indicating potential for faster and more cost-effective evaluation compared to expert assessments.
Usability describes a set of essential quality attributes of user interfaces (UI) that influence human-computer interaction. Common evaluation methods, such as usability testing and inspection, are effective but resource-intensive and require expert involvement. This makes them less accessible for smaller organizations. Recent advances in multimodal LLMs offer promising opportunities to automate usability evaluation processes partly by analyzing textual, visual, and structural aspects of software interfaces. To investigate this possibility, we formulate usability evaluation as a recommendation task, where multimodal LLMs rank usability issues by severity. We conducted an initial proof-of-concept study to compare LLM-generated usability improvement recommendations with usability expert assessments. Our findings indicate the potential of LLMs to enable faster and more cost-effective usability evaluation, which makes it a practical alternative in contexts with limited expert resources.