Active Learning for Robust and Representative LLM Generation in Safety-Critical Scenarios
This addresses safety-critical scenarios for user-facing systems by enhancing data representativeness, though it is incremental as it builds on existing active learning and clustering methods.
The paper tackled the problem of LLMs generating biased safety data by proposing an active learning framework with clustering, which produced a dataset of 5.4K safety violations and improved model accuracy and F1 scores.
Ensuring robust safety measures across a wide range of scenarios is crucial for user-facing systems. While Large Language Models (LLMs) can generate valuable data for safety measures, they often exhibit distributional biases, focusing on common scenarios and neglecting rare but critical cases. This can undermine the effectiveness of safety protocols developed using such data. To address this, we propose a novel framework that integrates active learning with clustering to guide LLM generation, enhancing their representativeness and robustness in safety scenarios. We demonstrate the effectiveness of our approach by constructing a dataset of 5.4K potential safety violations through an iterative process involving LLM generation and an active learner model's feedback. Our results show that the proposed framework produces a more representative set of safety scenarios without requiring prior knowledge of the underlying data distribution. Additionally, data acquired through our method improves the accuracy and F1 score of both the active learner model as well models outside the scope of active learning process, highlighting its broad applicability.