Evaluating LLM Agent Adherence to Hierarchical Safety Principles: A Lightweight Benchmark for Probing Foundational Controllability Components
This addresses the need for early detection of control deficiencies in AI safety plans, though it is incremental as it builds on existing safety verification methods.
The paper tackled the problem of verifying LLM agent adherence to safety principles when they conflict with task goals, by introducing a lightweight benchmark; the evaluation of six LLMs revealed a quantifiable 'cost of compliance' degrading task performance and an 'illusion of compliance' masking incompetence.
Credible safety plans for advanced AI development require methods to verify agent behavior and detect potential control deficiencies early. A fundamental aspect is ensuring agents adhere to safety-critical principles, especially when these conflict with operational goals. This paper introduces a lightweight, interpretable benchmark to evaluate an LLM agent's ability to uphold a high-level safety principle when faced with conflicting task instructions. Our evaluation of six LLMs reveals two primary findings: (1) a quantifiable "cost of compliance" where safety constraints degrade task performance even when compliant solutions exist, and (2) an "illusion of compliance" where high adherence often masks task incompetence rather than principled choice. These findings provide initial evidence that while LLMs can be influenced by hierarchical directives, current approaches lack the consistency required for reliable safety governance.