Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
This addresses the problem of AI safety for AI prototypers, offering a tool to mitigate harms during development, though it is incremental in enhancing existing prototyping workflows.
The paper tackles the challenge of identifying potential harms in AI applications during prompt-based prototyping by introducing Farsight, an interactive tool that helps users identify harms through news articles and LLM-generated content, resulting in improved independent harm identification and usability compared to existing resources.
Prompt-based interfaces for Large Language Models (LLMs) have made prototyping and building AI-powered applications easier than ever before. However, identifying potential harms that may arise from AI applications remains a challenge, particularly during prompt-based prototyping. To address this, we present Farsight, a novel in situ interactive tool that helps people identify potential harms from the AI applications they are prototyping. Based on a user's prompt, Farsight highlights news articles about relevant AI incidents and allows users to explore and edit LLM-generated use cases, stakeholders, and harms. We report design insights from a co-design study with 10 AI prototypers and findings from a user study with 42 AI prototypers. After using Farsight, AI prototypers in our user study are better able to independently identify potential harms associated with a prompt and find our tool more useful and usable than existing resources. Their qualitative feedback also highlights that Farsight encourages them to focus on end-users and think beyond immediate harms. We discuss these findings and reflect on their implications for designing AI prototyping experiences that meaningfully engage with AI harms. Farsight is publicly accessible at: https://PAIR-code.github.io/farsight.