fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks
This addresses the problem for UX designers in AI-driven design by providing a practical tool, though it is incremental as it builds on existing model card concepts.
The paper tackles the challenge for UX designers in understanding AI model limitations for computer vision tasks by introducing fAIlureNotes, a tool that supports failure exploration, and shows it outperforms interactive model cards in assessing context-specific performance.
To design with AI models, user experience (UX) designers must assess the fit between the model and user needs. Based on user research, they need to contextualize the model's behavior and potential failures within their product-specific data instances and user scenarios. However, our formative interviews with ten UX professionals revealed that such a proactive discovery of model limitations is challenging and time-intensive. Furthermore, designers often lack technical knowledge of AI and accessible exploration tools, which challenges their understanding of model capabilities and limitations. In this work, we introduced a failure-driven design approach to AI, a workflow that encourages designers to explore model behavior and failure patterns early in the design process. The implementation of fAIlureNotes, a designer-centered failure exploration and analysis tool, supports designers in evaluating models and identifying failures across diverse user groups and scenarios. Our evaluation with UX practitioners shows that fAIlureNotes outperforms today's interactive model cards in assessing context-specific model performance.