RUBAS: Rubric-Based Reinforcement Learning for Agent Safety
For developers of LLM-based agents, RUBAS provides a fine-grained, interpretable reward mechanism to balance safety and task completion in safety-critical tool-use settings.
RUBAS introduces a rubric-based reinforcement learning framework that decomposes agent behavior into four dimensions (tool-use safety, argument safety, response safety, helpfulness) to improve safety in LLM-based tool agents. Experiments show it reduces tool-grounded hallucinations and improves safety over standard alignment baselines while maintaining utility.
The evolution of LLMs into tool-enabled agents creates a new class of safety challenges associated with real-world execution rather than simple text generation. Existing alignment methods often rely on coarse refusal signals or static supervision, making it difficult to balance safety with useful tool execution across diverse agentic risks. We introduce RUBAS, a rubric-based reinforcement learning framework for agent safety. RUBAS decomposes agent behavior into four dimensions: tool-use safety, argument safety, response safety, and helpfulness. These structured rubrics provide fine-grained and interpretable rewards over complete agent trajectories, enabling reinforcement learning to optimize safe tool use while preserving task completion. Extensive experiments across multiple agent safety benchmarks and models show that RUBAS improves safety over standard alignment baselines, reduces tool-grounded hallucinations, and maintains competitive utility. Our results suggest that multi-dimensional rubric rewards provide an effective training signal for aligning LLM agents in safety-critical tool-use settings.