Safety Game: Balancing Safe and Informative Conversations with Blackbox Agentic AI using LP Solvers
This addresses the problem of inflexible and costly safety alignment for stakeholders, including smaller organizations, by providing a model-independent method, though it is incremental as it builds on inference-time alignment concepts.
The paper tackled the challenge of ensuring large language models comply with safety requirements without retraining or access to model internals by proposing a black-box framework that balances safe and informative conversations using a game-theoretic approach with linear programming solvers, demonstrating feasibility for scalable and accessible safety alignment.
Ensuring that large language models (LLMs) comply with safety requirements is a central challenge in AI deployment. Existing alignment approaches primarily operate during training, such as through fine-tuning or reinforcement learning from human feedback, but these methods are costly and inflexible, requiring retraining whenever new requirements arise. Recent efforts toward inference-time alignment mitigate some of these limitations but still assume access to model internals, which is impractical, and not suitable for third party stakeholders who do not have access to the models. In this work, we propose a model-independent, black-box framework for safety alignment that does not require retraining or access to the underlying LLM architecture. As a proof of concept, we address the problem of trading off between generating safe but uninformative answers versus helpful yet potentially risky ones. We formulate this dilemma as a two-player zero-sum game whose minimax equilibrium captures the optimal balance between safety and helpfulness. LLM agents operationalize this framework by leveraging a linear programming solver at inference time to compute equilibrium strategies. Our results demonstrate the feasibility of black-box safety alignment, offering a scalable and accessible pathway for stakeholders, including smaller organizations and entities in resource-constrained settings, to enforce safety across rapidly evolving LLM ecosystems.