To Err is AI : A Case Study Informing LLM Flaw Reporting Practices
This work addresses the problem of improving safety reporting processes for large language models, which is crucial for developers and users to prevent incidents, though it is incremental as it builds on existing bug bounty frameworks.
The paper analyzes a 2024 bug bounty program where 495 hackers evaluated the Open Language Model (OLMo) to identify flaws, leading to updates in documentation and cash awards, with the goal of establishing best practices for reporting LLM flaws to enhance safety.
In August of 2024, 495 hackers generated evaluations in an open-ended bug bounty targeting the Open Language Model (OLMo) from The Allen Institute for AI. A vendor panel staffed by representatives of OLMo's safety program adjudicated changes to OLMo's documentation and awarded cash bounties to participants who successfully demonstrated a need for public disclosure clarifying the intent, capacities, and hazards of model deployment. This paper presents a collection of lessons learned, illustrative of flaw reporting best practices intended to reduce the likelihood of incidents and produce safer large language models (LLMs). These include best practices for safety reporting processes, their artifacts, and safety program staffing.