HCJan 30, 2024
From Metrics to Meaning: Time to Rethink Evaluation in Human-AI Collaborative DesignSean P. Walton, Ben J. Evans, Alma A. M. Rahat et al.
As AI systems increasingly shape decision making in creative design contexts, understanding how humans engage with these tools has become a critical challenge for interactive intelligent systems research. This paper contributes a challenge to rethink how to evaluate human--AI collaborative systems, advocating for a more nuanced and multidimensional approach. Findings from one of the largest field studies to date (n = 808) of a human--AI co-creative system, The Genetic Car Designer, complemented by a controlled lab study (n = 12) are presented. The system is based on an interactive evolutionary algorithm where participants were tasked with designing a simple two dimensional representation of a car. Participants were exposed to galleries of design suggestions generated by an intelligent system, MAP--Elites, and a random control. Results indicate that exposure to galleries generated by MAP--Elites significantly enhanced both cognitive and behavioural engagement, leading to higher-quality design outcomes. Crucially for the wider community, the analysis reveals that conventional evaluation methods, which often focus on solely behavioural and design quality metrics, fail to capture the full spectrum of user engagement. By considering the human--AI design process as a changing emotional, behavioural and cognitive state of the designer, we propose evaluating human--AI systems holistically and considering intelligent systems as a core part of the user experience -- not simply a back end tool.
NAJul 9, 2025
An overlapping domain decomposition method for parametric Stokes and Stokes-Darcy problems via proper generalized decompositionMarco Discacciati, Ben J. Evans, Matteo Giacomini
A strategy to construct physics-based local surrogate models for parametric Stokes flows and coupled Stokes-Darcy systems is presented. The methodology relies on the proper generalized decomposition (PGD) method to reduce the dimensionality of the parametric flow fields and on an overlapping domain decomposition (DD) paradigm to reduce the number of globally coupled degrees of freedom in space. The DD-PGD approach provides a non-intrusive framework in which end-users only need access to the matrices arising from the (finite element) discretization of the full-order problems in the subdomains. The traces of the finite element functions used for the discretization within the subdomains are employed to impose arbitrary Dirichlet boundary conditions at the interface, without introducing auxiliary basis functions. The methodology is seamless to the choice of the discretization schemes in space, being compatible with both LBB-compliant finite element pairs and stabilized formulations, and the DD-PGD paradigm is transparent to the employed overlapping DD approach. The local surrogate models are glued together in the online phase by solving a parametric interface system to impose continuity of the subdomain solutions at the interfaces, without introducing Lagrange multipliers to enforce the continuity in the entire overlap and without solving any additional physical problem in the reduced space. Numerical results are presented for parametric single-physics (Stokes-Stokes) and multi-physics (Stokes-Darcy) systems, showcasing the accuracy, robustness, and computational efficiency of DD-PGD, and its capability to outperform DD methods based on high-fidelity finite element solvers in terms of computing times.