Mario Truss

Tim Rieder, Marian Schneider, Mario Truss et al.

A/B testing is a standard method for validating design decisions, yet its reliance on real user traffic limits iteration speed and makes certain experiments impractical. We present SimAB, a system that reframes A/B testing as a fast, privacy-preserving simulation using persona-conditioned AI agents. Given design screenshots and a conversion goal, SimAB generates user personas, deploys them as agents that state their preference, aggregates results, and synthesizes rationales. Through a formative study with experimentation practitioners, we identified scenarios where traffic constraints hinder testing, including low-traffic pages, multi-variant comparisons, micro-optimizations, and privacy-sensitive contexts. Our design emphasizes speed, early feedback, actionable rationales, and audience specification. We evaluate SimAB against 47 historical A/B tests with known outcomes, achieving 67% overall accuracy, increasing to 83% for high-confidence cases. Additional experiments show robustness to naming and positional bias and demonstrate accuracy gains from personas. Practitioner feedback suggests that SimAB supports faster evaluation cycles and rapid screening of designs difficult to assess with traditional A/B tests.

Mario Truss

LLM-based and agent-based synthetic personas are increasingly used in design and product decision-making, yet prior work shows that prompt-based personas often produce persuasive but unverifiable responses that obscure their evidentiary basis. We present PersonaCite, an agentic system that reframes AI personas as evidence-bounded research instruments through retrieval-augmented interaction. Unlike prior approaches that rely on prompt-based roleplaying, PersonaCite retrieves actual voice-of-customer artifacts during each conversation turn, constrains responses to retrieved evidence, explicitly abstains when evidence is missing, and provides response-level source attribution. Through semi-structured interviews and deployment study with 14 industry experts, we identify preliminary findings on perceived benefits, validity concerns, and design tensions, and propose Persona Provenance Cards as a documentation pattern for responsible AI persona use in human-centered design workflows.

Mario Truss, Marc Schmitt

This paper addresses the complexities inherent in AI product prototyping, focusing on the challenges posed by the probabilistic nature of AI behavior and the limited accessibility of prototyping tools to non-experts. A Design Science Research (DSR) approach is presented which culminates in a conceptual framework aimed at improving the AI prototyping process. Through a comprehensive literature review, key challenges were identified and no-code AutoML was analyzed as a solution. The framework describes the seamless incorporation of non-expert input and evaluation during prototyping, leveraging the potential of no-code AutoML to enhance accessibility and interpretability. A hybrid approach of combining naturalistic (case study) and artificial evaluation methods (criteria-based analysis) validated the utility of our approach, highlighting its efficacy in supporting AI non-experts and streamlining decision-making and its limitations. Implications for academia and industry, emphasizing the strategic integration of no-code AutoML to enhance AI product development processes, mitigate risks, and foster innovation, are discussed.

Mario Truss, Stephan Boehm

Automation of support ticket classification is crucial to improve customer support performance and shortening resolution time for customer inquiries. This research aims to test the applicability of automated machine learning (AutoML) as a technology to train a machine learning model (ML model) that can classify support tickets. The model evaluation conducted in this research shows that AutoML can be used to train ML models with good classification performance. Moreover, this paper fills a research gap by providing new insights into developing AI solutions without a dedicated professional by utilizing AutoML, which makes this technology more accessible for companies without specialized AI departments and staff.