Gerhard Leitner

Sebastian Lubos, Alexander Felfernig, Damian Garber et al.

Configuration is a key technology for tailoring complex software systems, services, and products. A successful application of configurators not only depends on technical correctness, performance, and domain modeling but also on their usability. While general usability heuristics are widely used, configurator-specific criteria and tool support for systematic user interface (UI) analysis are limited. This paper explores the use of multimodal large language models (MLLMs) for scalable and semi-automated usability analysis of configurator UIs. We synthesize 18 configurator-specific usability criteria from the literature and apply these criteria in an MLLM-based analysis of 16 real-world configurators. Each criterion is assessed individually to generate severity ratings for usability issues and actionable improvement suggestions. A review of the results confirms that MLLMs can reliably identify configurator-specific usability issues and provide domain-aware improvement recommendations. Although human validation remains necessary, this approach has the potential to significantly reduce the required effort to analyze configurator usability.

Hafsa Bousbiat, Christoph Klemenjak, Gerhard Leitner et al.

The need for reducing our energy consumption footprint and the increasing number of electric devices in today's homes is calling for new solutions that allow users to efficiently manage their energy consumption. Real-time feedback at device level would be of a significant benefit for this application. In addition, the aging population and their wish to be more autonomous have motivated the use of this same real-time data to indirectly monitor the household's occupants for their safety. By breaking down aggregate power consumption into its components, Non-Intrusive Load Monitoring provides information on individual appliances and their current state of operation. Since no additional metering equipment is required, residents are not confronted with intrusion into their familiar environment. Our work aims to depict an architecture supporting non-intrusive measurement with a smart electricity meter and the handling of these data using an open-source platform that allows to visualize and process real-time data about the total energy consumed. As a case study, we describe a series of measurements from common household devices and show how abnormal behavior can be detected.

Sebastian Lubos, Alexander Felfernig, Gerhard Leitner et al.

Usability describes a set of essential quality attributes of user interfaces (UI) that influence human-computer interaction. Common evaluation methods, such as usability testing and inspection, are effective but resource-intensive and require expert involvement. This makes them less accessible for smaller organizations. Recent advances in multimodal LLMs offer promising opportunities to automate usability evaluation processes partly by analyzing textual, visual, and structural aspects of software interfaces. To investigate this possibility, we formulate usability evaluation as a recommendation task, where multimodal LLMs rank usability issues by severity. We conducted an initial proof-of-concept study to compare LLM-generated usability improvement recommendations with usability expert assessments. Our findings indicate the potential of LLMs to enable faster and more cost-effective usability evaluation, which makes it a practical alternative in contexts with limited expert resources.

Martin Stettinger, Trang Tran, Ingo Pribik et al.

Existing e-learning environments primarily focus on the aspect of providing intuitive learning contents and to recommend learning units in a personalized fashion. The major focus of the KnowledgeCheckR environment is to take into account forgetting processes which immediately start after a learning unit has been completed. In this context, techniques are needed that are able to predict which learning units are the most relevant ones to be repeated in future learning sessions. In this paper, we provide an overview of the recommendation approaches integrated in KnowledgeCheckR. Examples thereof are utility-based recommendation that helps to identify learning contents to be repeated in the future, collaborative filtering approaches that help to implement session-based recommendation, and content-based recommendation that supports intelligent question answering. In order to show the applicability of the presented techniques, we provide an overview of the results of empirical studies that have been conducted in real-world scenarios.