Concrete Safety for ML Problems: System Safety for ML Development and Assessment
This addresses the problem of managing unforeseen risks in complex ML systems for stakeholders concerned with social harms and ethical violations, though it is incremental by adapting existing safety methods to ML.
The paper tackles the challenge of ensuring safety and trustworthiness in ML-driven systems by applying a state-of-the-art systems safety engineering approach to concrete applications with social and ethical risks, demonstrating a systematic method to meet assurance requirements for safe and trustworthy ML in sociotechnical systems.
Many stakeholders struggle to make reliances on ML-driven systems due to the risk of harm these systems may cause. Concerns of trustworthiness, unintended social harms, and unacceptable social and ethical violations undermine the promise of ML advancements. Moreover, such risks in complex ML-driven systems present a special challenge as they are often difficult to foresee, arising over periods of time, across populations, and at scale. These risks often arise not from poor ML development decisions or low performance directly but rather emerge through the interactions amongst ML development choices, the context of model use, environmental factors, and the effects of a model on its target. Systems safety engineering is an established discipline with a proven track record of identifying and managing risks even in high-complexity sociotechnical systems. In this work, we apply a state-of-the-art systems safety approach to concrete applications of ML with notable social and ethical risks to demonstrate a systematic means for meeting the assurance requirements needed to argue for safe and trustworthy ML in sociotechnical systems.